As a software company we are obsessed with automating our own processes to get things done faster. To have an edge against other competitors in our field (selling Odoo) we developed some features that we published in the Odoo store as apps. This requiered us to have a repository on GitHub. We use the same apps in our own client solutions and last month we published our PyPI repository. Keeping this many places up to date is something no one can or should do manually.
Luckily this was something we knew from the beginning when we started MuK IT. All repeatable processes that don’t need human interaction need to be automated. We are still not there with many parts of our company but software is the core of what we are selling. The goal was to develop faster by just concentrating on the development and fixing bugs instead of publishing and deploying. Combined with our Open Source strategy this gives us the ability to react faster to bugs and fix them everywhere with a commit. At least for us. Someone using this app still needs to download and upgrade it.
Everyone in software knows about Jenkins. It’s one of the most popular continous integration tools out there. Jenkins was our first approach to automate our software deployment. This totally failed for various reasons. For one it was expensive in resources and two I think it was too inconvenient at that time. Searching for solutions we tried the CI system of GitLab, our git repository. Having our CI tool in our git repository always present was the incentive we needed to focus on optimizing this process.
GitLab CI
Thanks to Gitlab we could move our CI from Jenkins (as we had planned) to our GIT repository. Saving us a lot of time, work and even real money.
At this time we tested software on our local machines. At first this wasn’t a problem but as we had more to do and more apps to maintain this got time consuming. We knew this was the first thing we need to get rid of. At the same time we were changing our infrastructure to kubernetes. So we wanted to combine this things. We wrote a docker image that we use for our clients and ourselves and we started testing against this image with everything installed that we use. When you push to the repository GitLab looks at a file called gitlab-ci.yml and decides what to do. In our case this was to tell kubernetes to run a container with our testimage and run the tests. After this is done GitLab knows if the test was successful or not. So this was our first simple CI pipeline.
The next thing we added was lint testing. For those who don’t know lint, lint is a tool that analyzes source code for errors and stylistic problems. Then came a publishing step and recently we added a step for testing security (with bandit). As resources cost money we had to decide what job gets triggered when. Our current strategy is that lint and security tests only get triggered if you say so in the commit and that the publishing works with tagging a version in our repository.
Here is some example code of our gitlab-ci.yml structure:
stages:
- lint
- bandit
- test
- publish
...
lint-python:
image: some.repository.com/odoo-test:11.0
stage: lint
only:
variables:
- $CI_COMMIT_MESSAGE =~ /\[lint\]/
script:
- pylint --load-plugins=pylint_odoo -e odoolint --disable=${PYLINT_DISABLE} --max-line-length=${PYLINT_MAX} ${CI_PROJECT_DIR}
allow_failure: true
As I’m a very big fan of this topic, probably a lot of software engineers are, there will be more on this in the future. We are always working on making our processes faster and better because this gives us the ability to concentrate on the more important things like talking to customers or developing features. If someone wants to get in touch with me to talk about this topic don’t hesitate to send me an email to kerrim.abdelhamed@mukit.at.