CI/CD Implementation and Testing

Glossary

Continuous Integration (CI): CI is a process in which developers integrate new codes into a shared repository several times a day. This is a better approach than the traditional one, where the developer will build new codes in isolation and then integrate into the main repository at the end of the project lifecycle. The primary goal is to detect any integration bugs at the initial stage so it can be rectified quickly. It triggers a new build whenever a new code merges with the existing main repository. Test runs are performed against these new builds to check for any breakage.

Continuous Delivery (CD): At the end of the CI, CD comes in. CD ensures to automate the software delivery process and commits to deliver the integrated code into the production stage without any bugs or delays. CD in the DevOps implementation process, helps developers merge the new code with the main branch consistently so they can build an instant software-ready product. It is responsible for checking the quality of the code and performs tests to check whether it can release the functional build to the production environment.

Continuous Deployment (CD): The most critical part of the automation occurs at this stage of the delivery pipeline. Whenever there is an important change in the code, the corresponding build and deployment occur simultaneously. It is achieved through a continuous deployment process where it allows to perform live deployment changes for every code modification that passes through the CI stage. During this stage, there is no manual intervention right from the initial code until the code is in the production stage.

Benefits for Implementing CI/CD

  • The ultimate aim of any software application is to reach its potential customers faster than ever before. This is exactly what the CI/CD pipeline offers to any business model that leverages a software process which reduces the risks in each build and helps the end product to reach the customer quicker.
  • Achieve Faster Feedback Through CI Tools: For every committed code, the corresponding tests are run simultaneously to avoid any link breakage at later stages. Faster feedback helps to check the quality and impact of the code.
  • Greater Visibility: With CI/CD pipeline setup, the whole process of new builds, test results, and any issues with new builds can be analyzed. This transparency allows the developers to know which changes in the build led to the brokerage and to avoid them in the future.
  • Early Bug detects: Conducting various types of automated testing allows us to identify any bug issues at an early stage without any last-minute surprises. These automated tests along with few manual test runs, help to fix any issues that may arise.

CI/CD processes in a project very much depend on the maturity of processes in the project as a whole and in testing in particular. Therefore, the first step with which we usually begin to improve the existing process is to evaluate the existing solution and build a baseline.

The baseline includes answers to the following questions:

Development:

  • What is the system architecture?
  • Which build server is used?
  • What is the principle of versioning?
  • How branching and merge strategy is defined?
  • How to define the build and release pipeline?
  • What is the Project Release creation strategy?
  • Are there any automatic code verification utilities?
  • Do developers write tests and are there utilities for evaluating code coverage by tests?
  • Are there integration tests and on what environment are they performed?

DevOps:

  • How many environments are deployed in the project and what is the strategy for their use?
  • How does the deployment happen?
  • Who has deployment privileges in test environments?
  • Who has access to the artifacts within the development, testing, and DevOps environments?
  • How is the environment being configured?
  • Who has access to the configuration of the environments?
  • What APIs of which systems are used in automated tests?
  • What are the data warehouses, external resources and who has access to them?
  • What third-party systems are used in the project?

Testing:

  • By what principle are testing levels divided?
  • How are Test Suites divided?
  • What is the test schedule?
  • How automated tests are written (technologies, incoming information, outgoing artifacts)?
  • How tests are integrated into the pipeline (steps, environments, suites)?
  • How is the coverage analysis performed?
  • How is auto-testing reporting performed?
  • How defects are reflected in autotests?
  • How is test data used?
  • How is test data created and updated (for functional and non-functional testing)?
  • What mocks are used in the testing process?
  • How is Hot-fix Testing Strategy defined?

Release:

  • What are the Quality Gates for releases?
  • How often do releases happen?
  • How are hot-fixes performed and delivered?
  • How is the release delivered to the customer?

Management:

  • What is written in the Test Plan document?
  • What are the Project  Acceptance Criteria?
  • How is the Traceability Matrix built?
  • How third-party system updates impact development and testing?
  • How is the integration testing with external systems performed?
  • What non-functional tests are carried out and with what frequency?
  • How is the management of sensitive data in the project performed?
  • What is the average lifetime of defects?
  • What is the location of defects?
  • How is test coverage by test levels distributed?

After the information is collected and analyzed, we offer 2-3 specific goals, the achievement of which will significantly improve the CI/CD processes in the project. At the same stage, metrics for assessing changes are discussed and approved.

To achieve these goals, we, together with all involved departments, draw up a plan, approve it and proceed to the implementation of the plan on the selected Pilot project: we introduce a new way of working either in a separate team or when developing specific functionality.

Upon completion of the Pilot project, we, first of all, assess whether the goals are achieved, make a retrospective, amend the plan of necessary actions and plan the implementation of the entire project.

At the end of the implementation, we conduct lessons learning and plan a periodic assessment of the project to make sure initial quality objectives are continuously met.


Related services:

Test project assessment

Test Project Assessment provides an independent and objective assessment of testing processes, provides the tools…