.. _plugin-testing-guide: *********************** Advanced Plugin Testing *********************** Why Write Tests? ================ `Automated testing `_ is a hallmark of modern software development. Adding a test suite to your plugin makes it easier to update and redesign your code, update its dependencies, etc. without having to manually check that every feature still works. Plugin Tests ============ First, we assume that you have already gone through the :ref:`Plugin Development Tutorial ` and created the plugin boilerplate with :command:`supybot-plugin-create`. Plugin Test Case Classes ------------------------ Limnoria comes with two plugin test case classes, :class:`supybot.test.PluginTestCase` and :class:`supybot.test.ChannelPluginTestCase`. Use the latter when the plugin's commands need to be run in a channel, and the former otherwise. Both of these classes inherit from Python's built-in `unittest.TestCase `_; as such, most of the documentation there applies to plugin tests too. Note that instead of ``python -m unittest``, Limnoria plugin tests are run using the :command:`limnoria-test` command: e.g. ``limnoria-test /path/to/your/Plugin`` Plugin Test Case Example ------------------------ A basic plugin test case requires: * The class declaration (subclassing one of the TestCase classes above) * A list of plugins to be loaded for these tests. Usually this is just the name of the plugin you're testing. Note that the **Owner**, **Misc**, and **Config** plugins are always automatically loaded * Some test methods :: class MyPluginTestCase(PluginTestCase): # List of plugins to load plugins = ('MyPlugin',) def testEcho(self): # Replace the command and expected response with your own, # add other assertions, etc. self.assertResponse('echo Hello world', 'Hello world') def testSomethingElse(self): # Add another test case here As with Python's unittest module in general, test methods must begin with ``test``. When adding helper methods in this class, they should start with something else. When writing your actual test methods, we recommend keeping each one short and targeted towards a specific feature. This will allow you to easily find which checks failed just by looking at a failed test's name, instead of having to sort through line numbers inside ``test.py``. To keep test functions unambiguous, it is fine to give them longer, more specific names to compensate (e.g. ``testOnlyRespondToRegisteredUsers`` instead of ``testRegistered``). Including Extra Setup --------------------- Plugin tests may define extra setup commands by overriding ``setUp()`` from Python's unittest module: .. code-block:: :dedent: 0 def setUp(self): # Important! This sets up the bot's simulated IRC network for testing super().setUp() # Define the identity of the user who we send messages as self.prefix = 'foo!bar@baz' # Send a message to the simulated IRC network, in this case to register # an account with the bot. self.nick refers to the bot's nick. self.feedMsg('register tester moo', to=self.nick, frm=self.prefix) m = self.getMsg(' ') # Get the response for the last command If your ``setUp`` function does work that should be cleaned up after, add a ``tearDown`` method as well. As with ``setUp``, you should also call the parent class' ``tearDown`` method *after* running your own cleanup. Setting Config Variables for Testing ------------------------------------ Config variables can be set at the test case level. For example, to disable nested commands for this test, you can add a ``config`` dict:: class MyPluginTestCase(PluginTestCase): config = {'supybot.commands.nested': False} def testThisThing(self): # stuff Temporarily setting a configuration variable -------------------------------------------- To temporarily set a config variable inside a test method, use the ``conf.supybot..context()`` context manager:: import supybot.conf as conf class MyPluginTestCase(PluginTestCase): def testThisThing(self): with conf.supybot.commands.nested.context(False): # stuff # when leaving the context manager, the config value is reverted to default .. _plugin-test-methods: Plugin Test Methods =================== In addition to Python's `built-in assertions `_, here are all the test methods defined in Limnoria. These are instance methods, so they should be accessed as ``self.assertResponse(...)``, etc. assertResponse(query, expectedResponse) Feeds query to the bot as a message and checks to make sure the response is expectedResponse. The test fails if they do not match (note that prefixed nicks in the response do not need to be included in the expectedResponse). assertError(query) Feeds query to the bot and expects an error in return. Fails if the bot doesn't return an error. assertNotError(query) The opposite of assertError. It doesn't matter what the response to query is, as long as it isn't an error. If it is not an error, this test passes, otherwise it fails. assertRegexp(query, regexp, flags=re.I) Feeds query to the bot and expects something matching the regexp (no m// required) in regexp with the supplied flags. Fails if the regexp does not match the bot's response. .. note:: This :func:`assertRegexp` function is `not` the same as :func:`assertRegex` from Python's unittest library. :func:`assertRegex` compares a regexp against a bare string, while :func:`assertRegexp` compares it to the output of a bot command. (For historical reasons, we have this confusing name.) assertNotRegexp(query, regexp, flags=re.I) The opposite of assertRegexp. Fails if the bot's output matches regexp with the supplied flags. assertHelp(query) Expects query to return the help for that command. Fails if the command help is not triggered. assertAction(query, expectedResponse=None) Feeds query to the bot and expects an action in response, specifically expectedResponse if it is supplied. Otherwise, the test passes for any action response. assertActionRegexp(query, regexp, flags=re.I) Basically like assertRegexp but carries the extra requirement that the response must be an action or the test will fail. Utilities --------- feedMsg(query, to=None, frm=None) Simply feeds query to whoever is specified in to or to the bot itself if no one is specified. Can also optionally specify the hostmask of the sender with the frm keyword. Does not actually perform any assertions. getMsg(query) Feeds query to the bot and gets the response. Tests for Helper Code ===================== If you want to test plugin helpers individually without running commands from your commands, you can add additional test classes inheriting from :class:`supybot.test.SupyTestCase`. This is a light wrapper around :class:`unittest.TestCase` that provides some additional logging. The **MoobotFactoids** plugin has an example of this (``OptionListTestCase``). The same rules for using ``setUp`` and ``tearDown`` apply: be sure to call the parent class implementations in your overridden functions.