Invalid Entry Infrastructure Architecture

The building of a community Discord bot grew rapidly into an entire, more formal definition of the Invalid Entry architecture.

Firstly, discord bots are easy to make, and its very tempting to make some kind of monolithic spaghetti code monster where the bot itself is doing everything, but I wanted to use this as a learning / teaching exercise about architecture.

Initially I knew I needed something to automate and manage my discord channel. I had previously written bots to do this on our friendly-work-social channel, where Invalid Entry had been sitting, but was outgrowing (or taking over) that original channel need. I also had a plan (with my friend ReasonDelta) to manage access via crypto fun madness, so the bot would be needed for that. The first stage is setting the scope.

Scope of architecture

So I started by thinking about the things I wanted the system to do, not necessarily in one go, but as an overall "it would be nice if one day it could" strategy. And also, super importantly, I wanted to make sure that we're focusing on 'Feature' as a thing it did, not how it did it, and so that list would look something like this:

It became apparent quickly that perhaps I was designing an architecture for the entire Invalid Entry home infrastructure, and thats fine.

The next thing was also considering what skills I had, and what skills I was willing to learn: Because this is a personal project, the scope of skills in this is limited to my own. In a professional sense, you would know about the skills of people in your organisation, but you also have to be aware of their commitment level. I.e. its fine to know that in some other team or department they have an expert in some system, but is that person going to be able to help you - can they cross train, will they be responsible for managing the system.

This list isn't something I wrote down (at the time), because, it only involved my self, but in a larger organisation its good to recognise what core skills we have, and then what secondary skills we gain from that, especially on a technology front.

The shortish list (of interest) would be: Python, Django, Databases, Queues, Docker etc.

This sets the scope for the technology, and lastly I wanted to set the scope of what benefits I wanted from the architecture:

1. I want to be able to develop discrete components over time, without breaking or affecting existing components. This suggests a microservice-ish approach.
2. I want to be able to develop individual services independently from others, i.e. I don't want a strong dependency between (for example) one thing running and working and another. This suggests a more strongly decoupled approach.
3. I've built a lot of Web-hook-API driven systems, and I wanted to try out some other approaches (e.g. inter-system queues or files etc).
4. Potentially this would be 'discord first' i.e. the primary way of interacting would be discord rather then webpages.
5. Non-Cloud/offline I wanted to run as much of the infrastructure on a pi as possible, to allow local interaction with science experiments etc. Obviously, Discord-bot would be online.

So the first draft of the architecture was primarily to bring up the discord bot. Breaking a bit of of the micro-service, I knew that it would be easier to have the discord bot tied into the database directly, as this would allow the discord-bot-first approach to be developed more.

Now comes the slightly more challenging aspect - deciding what to do first. I decided on a project plan of bringing the Discord bot up first to do community things, then the priority will be bringing the framework for discord up first, with other features being added secondary.

This is where the youtube series starts!

Database Development

The first steps are just getting the project setup and the initial database. Again, having future vision is important to be able to continue. The Database table layout is roughly as follows:

Adding the Bot Framework

The framework is important to get right. With this step it means future steps become easier.

The First useful command

So to validate that the bot framework is indeed working, making a command thats actually useful is important. This allows us to check that we can write to the db, and see that information in other channels.

Dockerisation

Dockerising this was more interesting. The decision to make a base image for the Django apps to inherit should mean that as I add more containers (and entry point jobs) they should all share the same images and free up some disk space. It also should reduce the overhead of some of the build steps.

Service Interaction

The final bit of the puzzle - here we want to start having those asynchronous interactions with the bot from hypothetical services.