Monthly KPI Report 2024 - Product Engineering Department
- Chathura Wijeratne
Introduction
At Camms, we are dedicated to fostering growth through continuous learning and improvement. In our Product Engineering division, we meticulously track a range of Key Performance Indicators (KPIs) to assess our delivery efficiency and ensure the best possible client experience. These KPIs serve as fundamental pillars in evaluating the performance of Camms products.
By leveraging these statistics, we aim to swiftly identify any potential risks and enhance our processes accordingly. As we advance, we will continuously refine and enhance these KPIs based on our observations and insights. Our commitment to leveraging data-driven insights enables us to drive continuous improvement and deliver exceptional value to our clients.
Formula / Base
Live Issues = Total number of open tickets at end of the respective month (From Kanban Teams Statistics)
Defect Leakage: Captured through QA Data set
Number of defects reported by end-users or customers (from Fresh Desk) X 100
Total number of defects found by QA (From JIRA)
Issues Injected Per Story Points
(Total number of Live Issues Reported for the Month + Total number of defects reported by BA+QA for the Month (JIRA)
Story points developed for the month and released
Development Velocity = Total Dev days available for the month * Total Story Points developed and Released
Total number of Dev Days worked Number of Developers
Story Point Calculation: 2Hrs of work = 1 Story Point (1 Block)
Length of a Sprint: 2 Weeks
Product KPIs: Year 2024
Overall Summary
SR | KPIs | Products | January | February | March | April | May | June | July | August | September | October | November | December |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | Live Issues - At end of each Month | Incident | 19 | 27 | 29 | 29 | 33 | 17 | 25 |
|
|
|
|
|
|
| Compliance | 4 | 7 | 11 | 7 | 11 | 9 | 2 |
|
|
|
|
|
|
| Controls | 8 | 8 | 7 | 11 | 6 | 4 | - |
|
|
|
|
|
|
| Risk | 29 | 57 | 48 | 37 | 27 | 9 | 27 |
|
|
|
|
|
|
| Audit | 11 | 13 | 13 | 11 | 4 | 1 | - |
|
|
|
|
|
|
| Survey | - | - | - | - | - | 2 | - |
|
|
|
|
|
|
| Mobile | 2 | 0 | 1 | 1 | 2 | 2 | 3 |
|
|
|
|
|
|
| Strategy | 13 | 13 | 13 | 15 | 12 | 7 | 9 |
|
|
|
|
|
|
| Project | 13 | 21 | 16 | 9 | 28 | 15 | 7 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2 | Defect Leakage | Incident | 14% | 28% | 53% | 20% | 46% | 27% | 25% |
|
|
|
|
|
|
| Compliance | - | - | - | - | 100% | - | 50% |
|
|
|
|
|
|
| Controls | - | - | - | - | - | - | - |
|
|
|
|
|
|
| Risk | 20% | 58% | 64% | 27% | 61% | 70% | 23% |
|
|
|
|
|
|
| Audit | - | - | - | - | - | - | - |
|
|
|
|
|
|
| Survey | - | - | - | - | - | - | - |
|
|
|
|
|
|
| Mobile | 0 | 0 | 0 | 8% | 9% | 0 | 0 |
|
|
|
|
|
|
| Strategy | 8% | 28% | 28% | 7% | 20% | 42% | 29% |
|
|
|
|
|
|
| Project | 7% | 21% | 44% | 7% | 23% | 32% | 15% |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
3 | Issues Injected per story point | Incident | 0 | 3.08 | 0.55 | 0.59 | 0.19 | 0.17 | 0.31 |
|
|
|
|
|
|
| Compliance | - | - | - | - | - | - | - |
|
|
|
|
|
|
| Controls | - | - | - | - | - | - | - |
|
|
|
|
|
|
| Risk | 0.85 | 1.10 | 0.31 | 0.34 | 0.29 | 0.20 | 0.37 |
|
|
|
|
|
|
| Audit | - | - | - | - | - | - | - |
|
|
|
|
|
|
| Mobile | 2.80 | 1.59 | 1.38 | 0.80 | 0.87 | 0.41 | 0.45 |
|
|
|
|
|
|
| Strategy | 2.35 | 1.60 | 0.28 | 0.60 | 0.52 | 0.16 | 0.19 |
|
|
|
|
|
|
| Project | - | - | - | - | - | 0.37 | 0.32 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
4 | Development Velocity | Incident | 0 | 25 | 58 | 59 | 23 | 58 | 48 |
|
|
|
|
|
|
| Compliance | - | - | - | - | - | - | - |
|
|
|
|
|
|
| Controls | - | - | - | - | - | - | - |
|
|
|
|
|
|
| Risk | 24 | 25 | 60 | 58 | 47 | 49 | 48 |
|
|
|
|
|
|
| Audit | - | - | - | - | - | - | - |
|
|
|
|
|
|
| Mobile | 23 | 24 | 23 | 58 | 58 | 60 | 35 |
|
|
|
|
|
|
| Strategy | 17 | 9 | 57 | 69 | 39 | 58 | 47 |
|
|
|
|
|
|
| Project | - | - | - | - | - | 46 | 59 |
|
|
|
|
|
Monthly KPIs - 2024
July 2024
Date Range: Jul 1, 2024 to Jul 31, 2024
KPIs | Incident | Compliance | Controls | Risk | Audit | Survey | Mobile | Strategy | Project |
|---|---|---|---|---|---|---|---|---|---|
Live Issues (<10)As of end of the month | 25 | 2 | - | 27 | - | - | 3 | 9 | 7 |
Defect Leakage Range: 5% - 15% | 25% | 50% | - | 23% | - | - | 0% | 29% | 15% |
Issues Injected Per Story Points Range: 1.5% - 5% | 0.31 | - | - | 0.37 | - | - | 0.45 | 0.19 | 0.32 |
Development Velocity Average Story Points developed by a Developer | 48 | - | - | 48 | - | - | 35 | 47 | 59 |
Notes |
| **Refer to the below comments | **No Developments was done for the month. | **Refer to the below comments | **No Developments was done for the month. | **No Developments was done for the month. |
|
| **Refer to the below comments |
Compliance Team - During the month there were no development was done. However, there were 2 Live Issues reported and 4 issues discovered while we were doing internal testing for Q3_FR_2024
June 2024
Date Range: Jun 1, 2024 to Jun 30, 2024
KPIs | Incident | Compliance | Controls | Risk | Audit | Survey | Mobile | Strategy | Project |
|---|---|---|---|---|---|---|---|---|---|
Live Issues (<10)As of end of the month | 17 | 9 | 4 | 9 | 1 | 2 | 2 | 7 | 15 |
Defect Leakage Range: 5% - 15% | 27% | - | - | 70% | - | - | 0% | 42% | 32% |
Issues Injected Per Story Points Range: 1.5% - 5% | 0.17 | - | - | 0.20 | - | - | 0.41 | 0.16 | 0.37 |
Development Velocity Average Story Points developed by a Developer | 58 | - | - | 49 | - | - | 60 | 58 | 46 |
Notes |
| **No Developments was done for the month. | **No Developments was done for the month. | **Refer to the below comments | **No Developments was done for the month. | **No Developments was done for the month. |
|
| **Refer to the below comments |
Risk Team - The Kanban team resolved 38 FD tickets, most of which were reported in previous months. Although the number of internally reported defects is low, our current formula indicates a higher figure. We will be revisiting and adjusting the formula in July.
Project Team - Some stories were delayed to release due to dependencies on external teams, resulting in a lower development velocity for June.
Project Team - Statistics for previous 5 months were not included, except Live Issues and Defect Leakages. With the transition to Product Engineering, the statistics will be captured from the month of June 2024.
May 2024
Date Range: May 1, 2024 to May 31, 2024
KPIs | Incident | Compliance | Controls | Risk | Audit | Mobile | Strategy |
|---|---|---|---|---|---|---|---|
Live Issues (<10)As of end of the month | 33 | 11 | 6 | 27 | 4 | 2 | 12 |
Defect Leakage Range: 5% - 15% | 46 | 100 | - | 61 | - | 9 | 20 |
Issues Injected Per Story Points Range: 1.5% - 5% | 0.19 | - | - | 0.29 | - | 0.87 | 0.52 |
Development Velocity Average Story Points developed by a Developer | 23 | - | - | 47 | - | 58 | 39 |
Notes | May has been a shorter month, and first couple of weeks were utilised for FR release. | **No Developments was done for the month. However team found 6 issues while executing the Regression suit for FR. | No Developments were done for the month May 2024. | Though it is a shorter month and having FR weeks, team worked on feature for Debeers during this period. | No Developments were done for the month May 2024. | May has been a shorter month, and first couple of weeks were utilised for FR release. Team continued to develop and release features related to Incident Revamp Application | May has been a shorter month, and first couple of weeks were utilised for FR release. |
The month of May was primarily dedicated to FR deployments, resulting in minimal development activities during the month. However, Risk and Mobile Team continued to develop based on their feature commitments.
April 2024
Date Range: Apr 1, 2024 to Apr 30, 2024
KPIs | Incident | Compliance | Controls | Risk | Audit | Mobile | Strategy |
|---|---|---|---|---|---|---|---|
Live Issues (<10)As of end of the month | 29 | 7 | 11 | 37 | 11 | 1 | 15 |
Defect Leakage Range: 5% - 15% | 20 | - | - | 27 | - | 8 | 7 |
Issues Injected Per Story Points Range: 1.5% - 5% | 0.59 | - | - | 0.34 | - | 0.80 | 0.60 |
Development Velocity Average Story Points developed by a Developer | 59 | - | - | 58 | - | 58 | 70 |
Notes | The teams started equating 1 story point to 2 hours from March onwards from the earlier 8 hours per story point. | **No Developments was done for the month. | **No Developments was done for the month. | The teams started equating 1 story points to 2 hours from March onwards from the earlier 8 hours per story point. | **No Developments was done for the month. | The teams started equating 1 story points to 2 hours from March onwards from the earlier 8 hours per story point. | The teams started equating 1 story points to 2 hours from March onwards from the earlier 8 hours per story point. |
The teams started equating 1 story point to 2 hours from March onwards from the earlier 8 hours per story point.
March 2024
Date Range: Mar 1, 2024 to Mar 31, 2024
KPIs | Incident | Compliance | Controls | Risk | Audit | Mobile | Strategy |
|---|---|---|---|---|---|---|---|
Live Issues (<10)As of end of the month | 29 | 11 | 7 | 48 | 13 | 1 | 13 |
Defect Leakage Range: 5% - 15% | 53 | - | - | 64 | - | 0 | 28 |
Issues Injected Per Story Points Range: 1.5% - 5% | 0.55 | - | - | 0.31 | - | 1.38 | 0.28 |
Development Velocity Average Story Points developed by a Developer | 58 | - | - | 60 | - | 23 | 57 |
Notes | The teams started equating 1 story point to 2 hours from March onwards from the earlier 8 hours per story point. | **No Developments was done for the month. | **No Developments was done for the month. | The teams started equating 1 story points to 2 hours from March onwards from the earlier 8 hours per story point. | **No Developments was done for the month. | **Refer to the below comments | The teams started equating 1 story points to 2 hours from March onwards from the earlier 8 hours per story point. |
The teams started equating 1 story point to 2 hours from March onwards from the earlier 8 hours per story point.
Mobile App Team - We've noticed a lower velocity, partly due to a significant number of stories spilling over into subsequent sprints. We consider story points completed only when the status is changed to "Sprint Review." We've identified several root causes for these spillages and are actively collaborating with the team to address them.
February 2024
Date Range: Feb 1, 2024 to Feb 29, 2024
KPIs | Incident | Compliance | Controls | Risk | Audit | Mobile | Strategy |
|---|---|---|---|---|---|---|---|
Live Issues (<10)As of end of the month | 27 | 7 | 8 | 57 | 13 | 0 | 13 |
Defect Leakage Range: 5% - 15% | 14 | - | - | 58 | - | - | 28 |
Issues Injected Per Story Points Range: 1.5% - 5% | 3.08 | - | - | 1.10 | - | 1.59 | 1.60 |
Development Velocity Average Story Points developed by a Developer | 25 | - | - | 25 | - | 24 | 9 |
Notes |
| **No Developments was done for the month. | **No Developments was done for the month. |
| **No Developments was done for the month. |
| **Refer to the below comments |
The month of February was primarily dedicated to FR deployments, resulting in minimal development activities during the month.
Strategy Team - commenced its development efforts on February 19, 2024, following the receipt of dependent deliveries required to initiate initial development. As a result, the number of completed story points during this period is minimal.
January 2024
Date Range: Jan 1, 2024 to Jan 31, 2024
KPIs | Incident | Compliance | Controls | Risk | Audit | Mobile | Strategy |
|---|---|---|---|---|---|---|---|
Live Issues (<10)As of end of the month | 19 | 4 | 8 | 29 | 11 | 2 | 13 |
Defect Leakage Range: 5% - 15% | 14 | - | - | 20 | - | 0 | 8 |
Issues Injected Per Story Points Range: 1.5% - 5% | 0 | - | - | 0.85 | - | 1.03 | 2.31 |
Development Velocity Average Story Points developed by a Developer | - | - | - | 24 | - | 23 | 17 |
Notes | **No Developments was done for the month. | **No Developments was done for the month. | **No Developments was done for the month. |
| **No Developments was done for the month. |
|
|
This month is dedicated to staging and testing releases. Consequently, the majority of the development work has been completed early.
Analysis of Trend
Year 2024
Monthly Trend
Development Velocity (Developer Average)- Monthly Trend
Month | Incident | Compliance | Controls | Risk | Audit | Mobile | Strategy | Project |
|---|---|---|---|---|---|---|---|---|
January | 0 | - | - | 24 | - | 23 | 17 | - |
February | 25 | - | - | 25 | - | 24 | 9 | - |
March | 58 | - | - | 60 | - | 23 | 56 | - |
April | 59 | - | - | 58 | - | 58 | 69 | - |
May | 23 | - | - | 47 | - | 58 | 39 | - |
June | 58 | - | - | 44 | - | 60 | 58 | 46 |
July | 48 | - | - | 48 | - | 35 | 47 | 57 |
Defect Leakage - Monthly Trend ( % )
Month | Incident | Compliance | Controls | Risk | Audit | Mobile | Strategy | Project |
|---|---|---|---|---|---|---|---|---|
January | 14 | - | - | 20 | - | 0 | 8 | 7 |
February | 15 | - | - | 58 | - | 0 | 28 | 21 |
March | 53 | - | - | 64 | - | 0 | 28 | 44 |
April | 20 | - | - | 27 | - | 8 | 7 | 7 |
May | 46 | 100 | - | 61 | - | 9 | 20 | 23 |
June | 27 | - | - | 70 | - | 0 | 42 | 32 |
July | 25 | 50 | - | 23 | - | 0 | 29 | 15 |
Team Velocity (Monthly Average)
Cumulative Running Totals
Defect Leakage - Cumulative Total ( % )
Month | Incident | Compliance | Controls | Risk | Audit | Survey | Mobile | Strategy | Project |
|---|---|---|---|---|---|---|---|---|---|
January | 14 | - | - | 20 | - | - | 0 | 8 | 7 |
February | 25 | - | - | 58 | - | - | 0 | 28 | 21 |
March | 32 | - | - | 60 | - | - | 0 | 28 | 29 |
April | 28 | - | - | 54 | - | - | 4 | 19 | 23 |
May | 32 | - | - | 55 | - | - | 6 | 19 | 23 |
June | 33 | - | - | 56 | - | - | 4 | 21 | 24 |
July | 32 | - | - | 50 | - | - | 3 | 21 | 22 |
Live Issues - Monthly trend
Month | Incident | Compliance | Controls | Risk | Audit | Survey | Mobile | Strategy | Project |
|---|---|---|---|---|---|---|---|---|---|
January | 19 | 4 | 8 | 29 | 11 | - | 2 | 13 | 13 |
February | 27 | 7 | 8 | 57 | 13 | - | 0 | 13 | 21 |
March | 29 | 11 | 7 | 48 | 13 | - | 1 | 13 | 16 |
April | 29 | 7 | 11 | 37 | 11 | - | 1 | 15 | 9 |
May | 33 | 11 | 6 | 27 | 4 | - | 2 | 12 | 28 |
June | 17 | 9 | 4 | 9 | 1 | 2 | 2 | 7 | 15 |
July | 25 | 2 | - | 27 | - | - | 3 | 9 | 7 |