#gitlab

Today, shit happened on a larger on-premise GitLab EE instance of one of our Onlime GmbH customers. GitLab's production.log started to fill up with PG::Error (FATAL: the database system is in recovery mode) errors which were somehow related to LFS operations. That definitely didn't sound cool and smelled like data corruption. The customer noticed it by failed CI jobs with 500 Internal Server Errors, and let me know immediately.

As we have that GitLab server running in a LXC container on a ZFS based system (Proxmox VE), it was easy to pull a clone of the full system and play around with PostgreSQL data recovery before working on live data. I decided to go for a full data restore by dumping and loading it from scratch in a freshly initialized PostgreSQL data dir.

Continuous deployment is great and a must for every modern web app. Forget about the times when you had to constantly log into your production server over SSH to run some git pull based deployment and cumbersome and error-prone build tasks, or building your project locally and then deploying it with rsync to production, which is not that sexy either. It is all doable and scriptable, but we want to have the whole process automated without any manual work involved. I much prefer to use GitLab CI/CD over GitHub Actions - ok, mainly just because I am more into it and prefer to run a self-hosted GitLab instance. GitLab just gets the job done very well!

We want the whole process to be straightforward without any fancy extras. It should be a matter of 15mins to set it up on every new project and I don't like to introduce any extra dependencies. I am just talking about deployment of a static site / SPA, a Vue.js based frontend that is generated by Nuxt.js. So let's keep it simple here! I am going to present you the solution I am using to deploy this TechBlog.