Howto

How to use a GitlabCI buildchain to ensure that your own TYPO3 project complies with the coding guidelines and deploy it fully automatically to the target server.

I've been a big fan of GitLab and its continuous integration solution since I got to know it a few years ago. In this Howto I would like to introduce how to realize your TYPO3 project in a CI buildchain and how to comply with the coding guidelines.

I am basically a big fan of automation. All jobs that a programmer has to do more than 3x belong to automation ;-) For this reason, my projects have a quality gate that controls the compliance with the coding guidelines and gives an alarm if a programmer tries to commit code that is not good enough.

 

Basic structure

As you can see in the picture, in a first step I let build the complete project, including dev-dependencies. Many of the other jobs depend on this state and it is much faster to build the complete project once than to let each job do it by itself - even if the jobs can run in parallel. 

The build stage is therefore essential for many of the subsequent stages: 

 

build application with dev dependencies:
  stage: build
  image: composer:1.9.0
  before_script:
    - apk add bash --no-cache
    - apk add git --update
  script:
    - composer global require hirak/prestissimo
    - composer install --no-progress --no-ansi --no-interaction
  artifacts:
    name: typo3
    paths:
      - ./vendor/
      - ./web/
      - ./composer.json
      - ./composer.lock
    expire_in: '1h'
  by day:
    - docker

 

I would like to draw your attention to the following line: this package causes composer to install all packages in parallel (instead of one after the other), thus speeding up the execution enormously!

 

composer global require hirak/prestissimo

 

 

Code Quality

In this stage, all jobs are executed that play a role in maintaining code quality. Exactly what these jobs are depends on the project and the quality requirements, of course.

TypoScript - Linter

TYPO3 projects actually always include TypoScript code. Because it is important for proper functioning, it belongs to the project versioned and not in the database. And because the code is versioned, it makes sense to give it a job in the quality gate to ensure quality. 

All kinds of things can go wrong with TypoScript code. Important in this context: TypoScript is not a programming language and it is not Turing-complete. This means, without wanting to go into detail: You can't calculate everything with TypoScript that can basically be calculated with a computer. But since TypoScript is mainly used to configure the TYPO3 system, this is fine and not a major problem.

Since TypoScript does not follow a flow chart, TypoScript itself cannot contain a syntax error; unknown statements are simply ignored. This is also something we have to keep in mind in this project: The linter does not (and in principle cannot!) check whether TypoScript does what it is supposed to do. It checks if brackets are set consistently and gives tips to keep things clear.

My TypoScript link stage looks like this: 

 

typoscript:
  stage: code quality
  image: composer:1.9.0
  script:
    - ./vendor/bin/typoscript-lint -c config/tslint.yaml
  tags:
    - docker

 

As you can also see here, I like to use docker containers on the GitlabCI server. To make the TypoScript-Linter available, it has to be installed once in the project: 

 

composer require --dev helmich/typo3-typoscript-lint

 

und steht dann sowohl auf der Entwicklermaschine als auch in der GitlabCI-Buildchain zur Verfügung.

Über eine YAML-Datei kann man den TypoScript-Lint auf die eigenen Bedürfnisse konfigurieren. Die Standardwerte sind durch die Bank sinnvoll gewählt; Änderungen daran lassen sich einfach konfigurieren: 

 

paths:
  - path/to/sitepackage/Configuration/TsConfig/
  - path/to/sitepackage/Configuration/TypoScript/

sniffs:
  - class: Indentation
    parameters:
      indentConditions: true
      useSpaces: true
      indentPerLevel: 2
  - class: RepeatingRValue
    disabled: true

 

Of course, it is recommended for all CodeQuality gates to run them on the development machine first and to correct any errors found.

Further information about the TypoScript-Linter: https://github.com/martin-helmich/typo3-typoscript-lint#configuration

SQL

In some projects I deploy SQL migrations. This is admittedly a checkered lily of the valley and probably unusual in most projects. But just such tasks should be checked by the quality gate to avoid accidentally playing out broken SQL. 

I will go into the migration in more detail at another time. For this article I will leave it at the configuration of the job: 

 

SQL:
  image: composer:1.9.0
  stage: code quality
  script:
    - vendor/bin/php-sqllint path/to/sql-files/*
  tags:
    - docker

 

and also here the package must be installed once:

 

composer require --dev cweiske/php-sqllint

 

 

Unit-Tests

Of course, we all write unit tests, or hopefully agree that it would make sense to have some. I like to write my tests with codeception, so of course the corresponding job in the build chain is designed for that. But it works the same way with PHPUnit or whatever you prefer as a test environment: 

 

unit tests:
  image: composer:1.9.0
  stage: code quality
  script:
    - vendor/bin/codecept run unit
  tags:
    - docker
  artifacts:
    name: tests
    paths:
      - ./tests/_output
    expire_in: 2 weeks
    when: always

 

Artefacts are also configured here. As a result, the output of the job (in this case, the results of the unit test) is made available for download in Gitlab-CI. In case a unit test turns red, this helps to narrow down the problem. 

Normally, this job should always turn green: as good developers, we have already run the tests locally and made sure we didn't break anything. 

YAML

YAML is becoming more and more an important configuration option in TYPO3. I'm not really happy with this file format, but I can't get around it anymore. 

This format is one of the few in my build chain that I don't test via PHP - maybe someone has a tip for me where to find a good YAML linter in PHP (this simplifies local execution on the development machine for me). 

 

yaml:
  image: python:alpine3.7
  stage: code quality
  before_script:
    - pip install yamllint==1.10.0
  script:
    - yamllint -c config/*.yaml .gitlab-ci.yml path/to/sitepackage/Configuration/Yaml
  tags:
    - docker

 

Besides the tools presented here, important YAML files are the CKEditor configurations and the TYPO3 forms.

PHPstan - static Code Analysis

Because we all don't always write perfect code, there are tools for us developers. An important one is phpstan - the static code analyzer. Phpstan works directly on the PHP source code, so it does not execute it. But it detects syntax errors, dead code and criticizes badly written or outdated code very well and very quickly. 

An excellent tool to make your code better and better. Phpstan knows different levels, with which you can increase your demands bit by bit. 

 

static code analysis:
  image: composer:1.9.0
  stage: code quality
  script:
    - vendor/bin/phpstan analyse -l 4 -c config/phpstan.neon
  tags:
    - docker

 

 and the corresponding configuration file

 

parameters:
  paths:
    - '../path/to/'
  ignoreErrors:
      - '#Constant TYPO3_MODE not found#'
      - '#Undefined variable: \$_EXTKEY#'

 

 

Reconstructor

 

Now comes my highlight: Especially if you have to re-invoice an older codebase, rector is a lifesaver. But I also like to use rector in new projects to prevent bad code from getting into the project at all. 

 

code review:
  image: composer:1.9.0
  stage: code quality
  script:
    - vendor/bin/rector process --config config/rector.yaml --dry-run
  tags:
    - docker

 

 Also rector must be installed once:

 

 composer require --dev rector/rector 

 

 

and brings along a configuration file: 

 

# rector.yaml
parameters:
  paths:
    - '/path/to'
  sets:
    - 'code-quality'
    - 'dead-code'
    - 'php73'

 

Which sets and settings you need for your project, you have to decide yourself. But Rector will provide you with really good and helpful suggestions to improve the code. 

On the development computer, you can look at them with 

 

vendor/bin/rector process --config config/rector.yaml --dry-run

 

and if desired change with 

 

vendor/bin/rector process --config config/rector.yaml

 

 

Prepare Deployment

When our change has come through the quality gate, we still have to prepare the code for delivery to the server. Since we have already built the system in the first stage including all dev-dependencies, the main task is to remove exactly those. Nothing easier than that: 

 

remove dev dependencies:
  stage: prepare for deployment
  image: composer:1.9.0
  before_script:
    - apk add bash --no-cache
    - apk add git --update
  script:
    - composer install --no-dev
  artifacts:
    name: typo3
    paths:
      - ./vendor/
      - ./web/
      - ./composer.json
      - ./composer.lock
    expire_in: '1h'
  tags:
    - docker

 

 

Frontend

Although I have built all the front-end assets on my development machine for development, I always want the front-end assets in my projects to be built on the Gitlab CI server before they are delivered. 

The advantage is that it is no longer possible for one team member to build the front-end assets and another team member cannot build them due to a version conflict in the build chain. 

At the same time, this way I make sure that not only the finished CSS and JavaScript, which has been assembled from several files, minified and possibly even obfuscated, is included in the repository. I want the repository to contain the source code files in SASS format, the properly built JavaScript files and the instructions on how to assemble them to the final result. 

This job depends heavily on how the frontend assets are built in your project. 

And deliver

Now everything is completely assembled, the code quality checked, the frontend assets created if necessary - we are ready. Next, the generated code must be transferred to the server.

There are many, many possibilities for this. You can use tools like Deployer, capistrano, TYPO3 surf and many more. You can transfer the generated data via FTP to the target host. 

Basically, all tools do the same thing: install data into a parallel directory, fire further necessary commands if necessary and when everything is ready, create a symlink from the previous version to the new version: the changes are played out and the new version is live.

By the way, if you do it right, you can do it with Zero Downtime. But more about that another time. 



Comments (0)

No comments found!

Write new comment