NEAL Home Page
Middlesex Logo

Code and Tutorial for Neural Text Generator

Code

Tutorial

  1. Get it running.
    1. Grab the tarball, unzip it, and untar it.
    2. I'm running in Ubuntu 16.04, with nest and spinnaker. You should get those running (or just nest if that's all you want).
    3. You should be able to build and test the neural language generation system in nest. You can build the system and test the generation of one sentence by entering the following in a shell: python testOneSentenceNest.py 2, where 2 refers to the sentence to test. The system should run error free, print out 'The yellow banana is in Room 1 2.' and generate three pkl files in the results directory. These pkl files represent the generation of the sentence (you can find more information about the pkl files in testOneSentenceNest.py).
    4. You can convert the pkl files to spike data using printGenResults.py. For example, enter: python printGenResults.py results/genSentence.pkl spikeSentenceStates 0, which prints the spikes, i.e. the spike times and the neurons that fired, to the screen and saves them to a text file called SpikeSentenceStates.sp. In this example, the spiking results show the state sequence of the generated sentence. You should see that neurons 190-197 are the last to fire; they finish spiking at about 245 ms (That's the final state of the sentence). The command should also print the sentence generated by the system. In this case it should print 'The yellow banana is in room 1 2.'. We note that the last part of the command above, i.e. 0, specifies whether you want a spike train plot or not, therefore, a 1 will produce one, a 0 will not.
  2. Explanation of Files
    1. All of the python files are for the pyNN neural language generator.
    2. chatbotLanguage.txt. This contains the language, i.e. the sentences our system will generate. See the readme file for more information on language files.
    3. generatorBaseClass.py: this is the base class for the neural language generation system. You should subclass it and use the subclass to build the topology of any given language (i.e. the sentences you want the system to generate).
    4. LanguageGenerator.py: this is a subclass of generatorBaseClass.py and builds the neural topology to generate the language defined in chatbotLanguage.txt. LanguageGenerator.py can be duplicated, modified and changed to your needs. testOneSentenceNest.py, testOneSentenceSpinn8.py SimpleChatbotAgentNest.py and SimpleChatbotAgentSpinn8.py use this
    5. testOneSentenceNest.py: this is described above.
    6. testOneSentenceSpinn8: this allows you to run the above on spinnaker (software from 2018).
    7. SimpleChatbotAgentNest.py and ChatBotInterfaceNest.py. These two files comprise a nest chatbot. The agent (chatbot) (coded in SimpleChatbotAgentNest.py) generates a response (i.e. a sentence from chatbotLanguage.txt) to queries provided by the interface (coded in ChatBotInterfaceNest.py).
    8. ChatBotInterfaceSpinn8.py and SimpleChatbotAgentSpinn8.py. These two files comprise a spinnaker chatbot.
    9. stateMachineClass.py: we use state machines, i.e. this class, to construct the sentences; it's well tested and should work robustly for generating sentences.
    10. nealCoverClass.py: neal is the Neuromophic Embodied Agents that Learn project. The idea is that we write agents by combining modules (like this language generator). Then we change one parameter (the simulator), and the system works on different platforms. Unfortunately some pyNN functions differ from simulator to simulator. These are included in nealCoverClass to reduce the amount of branching (e.g. if simulator == nest) in the modules.
    11. printGenResults.py: a program to convert pkl files generated from pyNN to spike data. It can also generate spiketrain plots.
    12. readme.txt contains brief information about the neural generation system, e.g. how to run it etc.
  3. Add a new sentence to the language generator.
    1. cp chatbotLanguage.txt chatbotLanguage2.txt
    2. cp testOneSentenceNest.py testOneSentence2Nest.py
    3. cp LanguageGenerator.py LanguageGenerator2.py
    4. modify the testOneSentence2Nest.py file to import LanguageGenerator2 instead of LanguageGenerator. Now:
      1. delete all .txt files apart from the chatbotLanguage and readme files
      2. run testOneSentence2Nest.py. It should run the same as testOneSentenceNest.
    5. We will now add the sentence 'The pineapple is badly damaged.' to the language generator. In chatbotLanguage2.txt, add 'The ?X is ?V.' under the line 'The ?X is in Room ?R.'. Under the line 'red apple' add 'pineapple'. Finally, under '1 2' add an empty line, then add '?V', and underneath add 'badly damaged'. See the readme file for more information on how to construct a language file.
    6. Running TestOneSentence2Nest.py should now build/include the new sentence in the generation system but currently you cannot activate/test it. To activate the new sentence:
      1. Open LanguageGenerator2.py and in function TestSentence copy elif(sentence==2): and all of the code within this elif statement and paste it underneath the code within the elif(sentence==3) statement. Now change the following in the pasted code:
        1. sentence==2 to sentence==4,
        2. Sentence="The ?X is in Room ?R." to Sentence="The ?X is ?V."
        3. phrase = "yellow banana" to phrase = "pineapple"
        4. Variable="?R" to Variable="?V"
        5. phrase = "1 2" to phrase= "badly damaged"
      2. Now, open testOneSentence2Nest.py and change the following: LanguageFile="chatbotLanguage.txt" to LanguageFile="chatbotLanguage2.txt"
    7. delete all .txt files apart from the chatbotLanguage and readme files
    8. run the test, i.e. python testOneSentence2Nest.py 4. Once the test has run, you should see the sentence 'The pineapple is badly damaged.' appear.
  4. Run the chatbot in nest.
    1. The code comes with a simple conversational agent (chatbot) and simple environment (interface). The Nest versions communicate via text files. To run the chatbot, do the following:
      1. Open up two shells.
      2. Remove all text files apart from the chatbotLanguage.txt and readme.txt files.
      3. Firstly run SimpleChatbotAgentNest.py in one shell., i.e. python SimpleChatbotAgentNest.py. In this shell you should see the timesteps of the simulation.
      4. In the other shell run ChatBotInterfaceNest.py, i.e. python ChatBotInterfaceNest.py. A prompt should appear asking you to enter a question, so type: where is the yellow banana? and hit return. You should then see 'query sent, waiting for response...'. Entering a question causes a query neuron in the agent to fire, which then activates a set of neurons, i.e. the answer to the question. Now check the output in the other shell (i.e. the one you are running SimpleChatbotAgentNest.py in). You should see the query you asked, the spike times and the neurons firing as the agent is processing a response. Now check the interface shell and you should eventually see the answer to the question, I.e. 'The yellow banana is in room 1 2'. Once the agent has answered the question, the interface will allow you to enter another question, e.g. where is the red apple?
  5. Modify the agent and interface in nest to use your new sentence as a new chatbot response.
    1. Modify the Agent
      1. Currently, the agent can only answer two questions, 'where is the yellow banana?' and 'where is the red apple?' (the answers to these are already specified in the code, which you can find in LanguageGenerator2.py functions TurnOnYellowBananaAnswerFromQuery and TurnOnRedAppleAnswerFromQuery.)
      2. We therefore want the agent to respond with 'The pineapple is badly damaged.', when given the new query 'What condition is the pineapple in?'.
      3. LanguageGenerator2.py:
        1. copy the whole function TurnOnRedAppleAnswerFromQuery, paste it underneath and change the following:
          1. Function Name TurnOnRedAppleAnswerFromQuery to TurnOnPineappleAnswerFromQuery
          2. Sentence="The ?X is in Room ?R." to Sentence="The ?X is ?V."
          3. phrase="red apple" to phrase="pineapple" under Variable "?X"
          4. Variable="?R" to Variable="?V"
          5. phrase="1 1" to phrase="badly damaged
        2. In function connectQueryCellsToAnswers:
          1. add Pquery=3 underneath Dquery=2
          2. copy self.TurnOnDiamondAnswerFromQuery(QueryCells,DQuery) and paste it underneath
          3. change the pasted code, i.e.: TurnOnDiamondAnswerFromQuery(QueryCells,DQuery) to TurnOnPineappleAnswerFromQuery(QueryCells,PQuery)
      4. cp SimpleChatbotAgentNest.py SimpleChatbotAgentNest2.py
      5. In SimpleChatbotAgentNest2.py change:
        1. 'from LanguageGenerator import GeneratorClass' to 'from LanguageGenerator2 import GeneratorClass
        2. find NumQueries=3 and change it to NumQueries=4
        3. In function RetrieveQuery, copy elif(query==2) and the code within the elif statement and paste it underneath. Now, change the newly pasted code, I.e.
          1. elif(query==2) to elif(query==3)
          2. "what shape is the diamond?" to "what condition is the pineapple in?"
        4. Find line LanguageFile="chatbotLanguage.txt" and change to LanguageFile="chatbotLanguage2.txt"
    2. Modify the Interface
      1. cp ChatBotInterfaceNest.py ChatBotInterfaceNest2.py
      2. In ChatBotInterfaceNest2.py: In the GetQueryFromUser() function, copy elif(query=="what shape is the diamond?") and the two lines within the elif statement and paste it underneath.
      3. Now change the newly pasted lines. Change (query=="what shape is the diamond?"): to (query=="what condition is the pineapple in?") and sendQuery(2) to sendQuery(3)
  6. Run The Modified Agent And Interface in nest.
    1. Before you run the agent and inteface remove all .txt files in the directory apart from the chatbotLanguage and readme text files.
    2. Open 2 shells.
    3. Firstly Run SimpleChatbotAgentNest2.py in one shell.
    4. Run ChatBotInterfaceNest2.py in the other shell. You should see a enter a question prompt, type: what condition is the pineapple in? and hit return.
    5. In the interface shell you should eventually see 'The pineapple is badly damaged.'. If you want to look at the spike trains generated, convert the pkl files (as explained earlier).
This page was developed by Chris Huyck
Page last modified