CI/CD Implementation and Testing

CI/CD processes in a project very much depend on the maturity of processes in the project as a whole and in testing in particular. Therefore, the first step with which we usually begin to improve the existing process is to evaluate the existing solution and build a baseline.

The baseline includes answers to the following questions:

Development:

  • What is the system architecture?
  • Which build server is used?
  • What is the principle of versioning?
  • How branching and merge strategy is defined?
  • How to define the build and release pipeline?
  • What is the Project Release creation strategy?
  • Are there any automatic code verification utilities?
  • Do developers write tests and are there utilities for evaluating code coverage by tests?
  • Are there integration tests and on what environment are they performed?

DevOps:

  • How many environments are deployed in the project and what is the strategy for their use?
  • How does the deployment happen?
  • Who has deployment privileges in test environments?
  • Who has access to the artifacts within the development, testing, and DevOps environments?
  • How is the environment being configured?
  • Who has access to the configuration of the environments?
  • What APIs of which systems are used in automated tests?
  • What are the data warehouses, external resources and who has access to them?
  • What third-party systems are used in the project?

Testing:

  • By what principle are testing levels divided?
  • How are Test Suites divided?
  • What is the test schedule?
  • How automated tests are written (technologies, incoming information, outgoing artifacts)?
  • How tests are integrated into the pipeline (steps, environments, suites)?
  • How is the coverage analysis performed?
  • How is auto-testing reporting performed?
  • How defects are reflected in autotests?
  • How is test data used?
  • How is test data created and updated (for functional and non-functional testing)?
  • What mocks are used in the testing process?
  • How is Hot-fix Testing Strategy defined?

Release:

  • What are the Quality Gates for releases?
  • How often do releases happen?
  • How are hot-fixes performed and delivered?
  • How is the release delivered to the customer?

Management:

  • What is written in the Test Plan document?
  • What are the Project  Acceptance Criteria?
  • How is the Traceability Matrix built?
  • How third-party system updates impact development and testing?
  • How is the integration testing with external systems performed?
  • What non-functional tests are carried out and with what frequency?
  • How is the management of sensitive data in the project performed?
  • What is the average lifetime of defects?
  • What is the location of defects?
  • How is test coverage by test levels distributed?

After the information is collected and analyzed, we offer 2-3 specific goals, the achievement of which will significantly improve the CI/CD processes in the project. At the same stage, metrics for assessing changes are discussed and approved.

To achieve these goals, we, together with all involved departments, draw up a plan, approve it and proceed to the implementation of the plan on the selected Pilot project: we introduce a new way of working either in a separate team or when developing specific functionality.

Upon completion of the Pilot project, we, first of all, assess whether the goals are achieved, make a retrospective, amend the plan of necessary actions and plan the implementation of the entire project.

At the end of the implementation, we conduct lessons learning and plan a periodic assessment of the project to make sure initial quality objectives are continuously met.


Related services:

Test project assessment

Test Project Assessment provides an independent and objective assessment of testing processes, provides the tools…