Rig up first-class file system support

See https://git.vvvvvvaria.org/decentral1se/xbotlib/issues/3.

.gitignore

@@ -1,5 +1,6 @@
 *.conf
 *.egg-info/
+*.json
 *.pyc
 .avatars
 .coverage

README.md

@@ -36,7 +36,7 @@ you want to chat or just invite your bots for testing.
     - [Using the `.conf` configuration file](#using-the--conf--configuration-file)
     - [Using the command-line interface](#using-the-command-line-interface)
     - [Using the environment](#using-the-environment)
-  - [Persistent storage](#persistent-storage)
+  - [Storage back-end](#storage-back-end)
     - [File system](#file-system)
     - [Redis key/value storage](#redis-key-value-storage)
   - [Loading Plugins](#loading-plugins)
@@ -255,46 +255,35 @@ deployments.
 - **XBOT_STORAGE**: choice of storage back-end (default: `file`)
 - **XBOT_STORAGE_FILE**: path to file based storage back-end (default: `<nick>.json`)
 
-### Persistent storage
+### Storage back-end
 
-#### File system
+In order to store data you can make use of the `self.db` attribute of the `Bot`
-
+class. It is a Python dictionary which will be saved to disk automatically for
-Just use your local file system as you would in any other Python script. Please
+you as a `<nick>.json` in your current working directory. The name and path to
-note that when you deploy your bot, you might not have access to this local
+this file can be configured.
-filesystem in the same location. For remote server deployments
-[Redis](#redis-key-value-storage) can be more convenient.
-
-#### Redis key/value storage
-
-`xbotlib` supports using [Redis](https://redis.io/) as a storage back-end. It
-is simple to work with because the interface is exactly like a dictionary. You
-can quickly run Redis locally using [Docker](https://docs.docker.com/engine/install/debian/)
-(`docker run --network=host --name redis -d redis`) or if you're on a Debian system you can
-also `sudo apt install -y redis`.
-
-You can configure the connection URL using the command-line interface,
-configuration or environment. Here is an example using the environment.
-
-```bash
-$ export XBOT_REDIS_URL=redis://localhost:6379/0
-```
-
-And you access the interface via the `self.db` attribute.
 
 ```python
-def direct(self, message):
+def group(self, message):
-    self.db["mykey"] = message.text
+    if not message.room in self.db.keys():
+        self.db[message.room] = "visited"
 ```
 
-You should see `INFO Successfully connected to storage` when your bot
+If you want to inspect the database when the bot is not running, you can look
-initialises. Please see the
+in the file directly.
-[redis-py](https://redis-py.readthedocs.io/en/stable/) API documentation for
+
-more.
+```bash
+$ cat mybot.json
+```
+
+For more advanced use cases, `xbotlib` also supports [Redis](https://redis.io/)
+as a storage back-end. You'll need to configure this (e.g. `--storage redis`)
+as the default uses the filesystem approach mentioned above. The same `self.db`
+will then be passed as a Redis connection object.
 
 ### Loading Plugins
 
 You can specify a `plugins = [...]` on your bot definition and they will be
-automatically loaded.
+automatically loaded when you start your bot.
 
 ```python
 class MyBot(Bot):

xbotlib.py

@@ -7,6 +7,7 @@ from datetime import datetime as dt
 from getpass import getpass
 from imghdr import what
 from inspect import cleandoc
+from json import dumps, loads
 from logging import DEBUG, INFO, basicConfig, getLogger
 from os import environ
 from os.path import exists
@@ -155,6 +156,16 @@ class Config:
         """Turn on the web server."""
         return self.section.get("serve", None)
 
+    @property
+    def storage(self):
+        """Choice of storage back-end."""
+        return self.section.get("storage", None)
+
+    @property
+    def storage_file(self):
+        """Path to the file based storage back-end."""
+        return self.section.get("storage_file", None)
+
 
 class Bot(ClientXMPP):
     """XMPP bots for humans."""
@@ -174,7 +185,7 @@ class Bot(ClientXMPP):
         self.init_bot()
         self.register_xmpp_event_handlers()
         self.register_xmpp_plugins()
-        self.init_db()
+        self.init_storage()
         self.run()
 
     def parse_arguments(self):
@@ -252,6 +263,19 @@ class Bot(ClientXMPP):
             dest="serve",
             help="turn on the web server",
         )
+        self.parser.add_argument(
+            "-st",
+            "--storage",
+            dest="storage",
+            help="choice of storage back-end",
+            choices=("file", "redis"),
+        )
+        self.parser.add_argument(
+            "-stf",
+            "--storage-file",
+            dest="storage_file",
+            help="path to file based storage back-end",
+        )
 
         self.args = self.parser.parse_args()
 
@@ -377,6 +401,18 @@ class Bot(ClientXMPP):
             or self.config.serve
             or environ.get("XBOT_SERVE", None)
         )
+        storage = (
+            self.args.storage
+            or self.config.storage
+            or environ.get("XBOT_STORAGE", None)
+            or "file"
+        )
+        storage_file = (
+            self.args.storage_file
+            or self.config.storage_file
+            or environ.get("XBOT_STORAGE_FILE", None)
+            or f"{nick}.json"
+        )
 
         if not account:
             self.log.error("Unable to discover account")
@@ -400,6 +436,8 @@ class Bot(ClientXMPP):
         self.port = port
         self.template = self.load_template(template)
         self.serve = serve
+        self.storage = storage
+        self.storage_file = Path(storage_file).absolute()
 
     def load_template(self, template):
         """Load template via Jinja."""
@@ -526,17 +564,26 @@ class Bot(ClientXMPP):
         except AttributeError:
             self.log.info("No additional plugins loaded")
 
-    def init_db(self):
+    def init_storage(self):
-        """Initialise the Redis key/value store."""
+        """Initialise the storage back-end."""
-        if not self.redis_url:
+        if self.storage == "file":
-            self.db = None
+            try:
-            return self.log.info("No Redis storage discovered")
+                self.db = {}
-
+                if exists(self.storage_file):
-        try:
+                    self.db = loads(open(self.storage_file, "r").read())
-            self.db = Redis.from_url(self.redis_url, decode_responses=True)
+                self.log.info("Successfully loaded file storage")
-            self.log.info("Successfully connected to Redis storage")
+            except Exception as exception:
-        except ValueError:
+                message = f"Failed to load {self.storage_file}: {exception}"
-            self.log.info("Failed to connect to Redis storage")
+                self.log.info(message)
+                exit(1)
+        else:
+            try:
+                self.db = Redis.from_url(self.redis_url, decode_responses=True)
+                return self.log.info("Successfully connected to Redis storage")
+            except ValueError as exception:
+                message = f"Failed to connect to Redis storage: {exception}"
+                self.log.info(message)
+                exit(1)
 
     def run(self):
         """Run the bot."""
@@ -548,7 +595,17 @@ class Bot(ClientXMPP):
                 self.serve_web()
             self.process(forever=False)
         except (KeyboardInterrupt, RuntimeError):
-            pass
+            if self.storage != "file":
+                exit(0)
+
+            try:
+                with open(self.storage_file, "w") as handle:
+                    handle.write(dumps(self.db))
+                self.log.info("Successfully saved file storage")
+            except Exception as exception:
+                message = f"Failed to save file storage: {exception}"
+                self.log.info(message)
+                exit(1)
 
     def serve_web(self):
         """Serve the web."""