Wolf Paulus (which is a fabulous name in itself) has written a comprehensive tutorial which teaches you how to install SphinxBase and PocketSphinx on the Pi and have it perform speech recognition. As Wolf points out, you can’t expect it to perform as fast as something like Google’s recogniser. Read it here.