I've been a big fan of GitLab and its continuous integration solution since I got to know it a few years ago. In this Howto I would like to introduce how to realize your TYPO3 project in a CI buildchain and how to comply with the coding guidelines.
I am basically a big fan of automation. All jobs that a programmer has to do more than 3x belong to automation ;-) For this reason, my projects have a quality gate that controls the compliance with the coding guidelines and gives an alarm if a programmer tries to commit code that is not good enough.
As you can see in the picture, in a first step I let build the complete project, including dev-dependencies. Many of the other jobs depend on this state and it is much faster to build the complete project once than to let each job do it by itself - even if the jobs can run in parallel.
The build stage is therefore essential for many of the subsequent stages:
build application with dev dependencies: stage: build image: composer:1.9.0 before_script: - apk add bash --no-cache - apk add git --update script: - composer global require hirak/prestissimo - composer install --no-progress --no-ansi --no-interaction artifacts: name: typo3 paths: - ./vendor/ - ./web/ - ./composer.json - ./composer.lock expire_in: '1h' by day: - docker
I would like to draw your attention to the following line: this package causes composer to install all packages in parallel (instead of one after the other), thus speeding up the execution enormously!
composer global require hirak/prestissimo
In this stage, all jobs are executed that play a role in maintaining code quality. Exactly what these jobs are depends on the project and the quality requirements, of course.
TypoScript - Linter
TYPO3 projects actually always include TypoScript code. Because it is important for proper functioning, it belongs to the project versioned and not in the database. And because the code is versioned, it makes sense to give it a job in the quality gate to ensure quality.
All kinds of things can go wrong with TypoScript code. Important in this context: TypoScript is not a programming language and it is not Turing-complete. This means, without wanting to go into detail: You can't calculate everything with TypoScript that can basically be calculated with a computer. But since TypoScript is mainly used to configure the TYPO3 system, this is fine and not a major problem.
Since TypoScript does not follow a flow chart, TypoScript itself cannot contain a syntax error; unknown statements are simply ignored. This is also something we have to keep in mind in this project: The linter does not (and in principle cannot!) check whether TypoScript does what it is supposed to do. It checks if brackets are set consistently and gives tips to keep things clear.
My TypoScript link stage looks like this:
typoscript: stage: code quality image: composer:1.9.0 script: - ./vendor/bin/typoscript-lint -c config/tslint.yaml tags: - docker
As you can also see here, I like to use docker containers on the GitlabCI server. To make the TypoScript-Linter available, it has to be installed once in the project:
composer require --dev helmich/typo3-typoscript-lint
und steht dann sowohl auf der Entwicklermaschine als auch in der GitlabCI-Buildchain zur Verfügung.
Über eine YAML-Datei kann man den TypoScript-Lint auf die eigenen Bedürfnisse konfigurieren. Die Standardwerte sind durch die Bank sinnvoll gewählt; Änderungen daran lassen sich einfach konfigurieren:
paths: - path/to/sitepackage/Configuration/TsConfig/ - path/to/sitepackage/Configuration/TypoScript/ sniffs: - class: Indentation parameters: indentConditions: true useSpaces: true indentPerLevel: 2 - class: RepeatingRValue disabled: true
Of course, it is recommended for all CodeQuality gates to run them on the development machine first and to correct any errors found.
Further information about the TypoScript-Linter: https://github.com/martin-helmich/typo3-typoscript-lint#configuration
In some projects I deploy SQL migrations. This is admittedly a checkered lily of the valley and probably unusual in most projects. But just such tasks should be checked by the quality gate to avoid accidentally playing out broken SQL.
I will go into the migration in more detail at another time. For this article I will leave it at the configuration of the job:
SQL: image: composer:1.9.0 stage: code quality script: - vendor/bin/php-sqllint path/to/sql-files/* tags: - docker
and also here the package must be installed once:
composer require --dev cweiske/php-sqllint
Of course, we all write unit tests, or hopefully agree that it would make sense to have some. I like to write my tests with codeception, so of course the corresponding job in the build chain is designed for that. But it works the same way with PHPUnit or whatever you prefer as a test environment:
unit tests: image: composer:1.9.0 stage: code quality script: - vendor/bin/codecept run unit tags: - docker artifacts: name: tests paths: - ./tests/_output expire_in: 2 weeks when: always
Artefacts are also configured here. As a result, the output of the job (in this case, the results of the unit test) is made available for download in Gitlab-CI. In case a unit test turns red, this helps to narrow down the problem.
Normally, this job should always turn green: as good developers, we have already run the tests locally and made sure we didn't break anything.
YAML is becoming more and more an important configuration option in TYPO3. I'm not really happy with this file format, but I can't get around it anymore.
This format is one of the few in my build chain that I don't test via PHP - maybe someone has a tip for me where to find a good YAML linter in PHP (this simplifies local execution on the development machine for me).
yaml: image: python:alpine3.7 stage: code quality before_script: - pip install yamllint==1.10.0 script: - yamllint -c config/*.yaml .gitlab-ci.yml path/to/sitepackage/Configuration/Yaml tags: - docker
Besides the tools presented here, important YAML files are the CKEditor configurations and the TYPO3 forms.
PHPstan - static Code Analysis
Because we all don't always write perfect code, there are tools for us developers. An important one is phpstan - the static code analyzer. Phpstan works directly on the PHP source code, so it does not execute it. But it detects syntax errors, dead code and criticizes badly written or outdated code very well and very quickly.
An excellent tool to make your code better and better. Phpstan knows different levels, with which you can increase your demands bit by bit.
static code analysis: image: composer:1.9.0 stage: code quality script: - vendor/bin/phpstan analyse -l 4 -c config/phpstan.neon tags: - docker
and the corresponding configuration file
parameters: paths: - '../path/to/' ignoreErrors: - '#Constant TYPO3_MODE not found#' - '#Undefined variable: \$_EXTKEY#'
Now comes my highlight: Especially if you have to re-invoice an older codebase, rector is a lifesaver. But I also like to use rector in new projects to prevent bad code from getting into the project at all.
code review: image: composer:1.9.0 stage: code quality script: - vendor/bin/rector process --config config/rector.yaml --dry-run tags: - docker
Also rector must be installed once:
composer require --dev rector/rector
and brings along a configuration file:
# rector.yaml parameters: paths: - '/path/to' sets: - 'code-quality' - 'dead-code' - 'php73'
Which sets and settings you need for your project, you have to decide yourself. But Rector will provide you with really good and helpful suggestions to improve the code.
On the development computer, you can look at them with
vendor/bin/rector process --config config/rector.yaml --dry-run
and if desired change with
vendor/bin/rector process --config config/rector.yaml
When our change has come through the quality gate, we still have to prepare the code for delivery to the server. Since we have already built the system in the first stage including all dev-dependencies, the main task is to remove exactly those. Nothing easier than that:
remove dev dependencies: stage: prepare for deployment image: composer:1.9.0 before_script: - apk add bash --no-cache - apk add git --update script: - composer install --no-dev artifacts: name: typo3 paths: - ./vendor/ - ./web/ - ./composer.json - ./composer.lock expire_in: '1h' tags: - docker
Although I have built all the front-end assets on my development machine for development, I always want the front-end assets in my projects to be built on the Gitlab CI server before they are delivered.
The advantage is that it is no longer possible for one team member to build the front-end assets and another team member cannot build them due to a version conflict in the build chain.
This job depends heavily on how the frontend assets are built in your project.
Now everything is completely assembled, the code quality checked, the frontend assets created if necessary - we are ready. Next, the generated code must be transferred to the server.
There are many, many possibilities for this. You can use tools like Deployer, capistrano, TYPO3 surf and many more. You can transfer the generated data via FTP to the target host.
Basically, all tools do the same thing: install data into a parallel directory, fire further necessary commands if necessary and when everything is ready, create a symlink from the previous version to the new version: the changes are played out and the new version is live.
By the way, if you do it right, you can do it with Zero Downtime. But more about that another time.