The Internal Defect Ranker helps engineering teams identify the highest-value backlog defects to address within the engineering capacity reserved for defect fixing during a sprint.

It evaluates open backlog bugs across priority, age, scoring objectives, and estimated engineering effort, and produces a capacity-aware recommendation of which defects should be addressed first.

The agent is designed for Scrum Masters, Engineering Managers, and Engineering Leads who want to make defect triage more evidence-driven instead of relying only on ticket priority or backlog order.

What does the Internal Defect Ranker do?

Engineering backlogs can contain more defects than the team has capacity to address in a sprint. Determining which defects provide the best use of the available defect-fixing capacity can therefore be difficult.

The Internal Defect Ranker evaluates eligible backlog bugs and determines which combination provides the best overall fit within the configured engineering capacity.

The agent automatically:

1. Identifies eligible open bugs from the connected planning system.

2. Normalises the backlog items into a common structure.

3. Establishes a baseline effort estimate for each bug using available estimation signals.

4. Scores bugs using priority, age, and configured scoring objectives.

5. Ranks the bugs based on their overall score.

6. Uses available code and story context to refine estimates for the highest-ranked candidates.

7. Selects the best-fit combination of defects within the configured engineering capacity.

8. Identifies important defects that could not fit within the available capacity.

9. Generates a triage report explaining the recommendations.

10. Sends the report to configured recipients when email delivery is enabled.

In simple terms: Internal Defect Ranker helps engineering teams decide which backlog defects are most worth spending engineering capacity on now.

Before you start

For the agent to produce meaningful recommendations, the workspace should have:

• A connected planning tool containing the engineering backlog.

• Open bugs available in Cubyts.

• A configured engineering capacity or bug budget.

• Appropriate scoring guidance when specific defect-impact objectives need to receive additional consideration.

• A coding integration when code-context-based estimate refinement is required.

• Appropriate workspace members configured as the audience.

• Email recipients configured if email delivery is required.

The agent can also operate with ADO as the planning system where configured.

Configure Internal Defect Ranker

To configure the agent, open:

Settings → Agents → Internal Defect Ranker

The configuration consists of three sections:

1. Run Configurations

2. Audience

3. Integrations

The agent can be enabled or disabled using the toggle in the top-right corner.

The agent is intended for:

• Scrum Master

• Engineering Manager

• Engineering Lead

1. Run Configurations

The Run Configurations section controls when the agent runs and how backlog bugs are evaluated.

Scheduler

Use the Scheduler toggle to enable or disable automatic execution of the agent.

When enabled, you can select the required run frequency:

• Daily

• Weekly

• Monthly

For a weekly schedule, select the day on which the audit should run.

For example:

Weekly → Wednesday

The agent runs at midnight for the configured schedule.

Bug budget (hours)

Bug budget (hours) defines how many engineering hours are available for defect fixing during the run.

For example, if the bug budget is configured as 19 hours, the agent attempts to select the combination of bugs that provides the highest overall value while staying within those 19 hours.

The implementation supports a configurable range of 1–1000 hours, with a documented default of 40 hours.

The budget is important because the agent does not simply select the highest-scoring bugs individually. It considers the combination of bugs that provides the best overall score within the available capacity.

Bug work item type 

Bug work item type specifies which work item type the agent should evaluate in the planning system.

The default value is: Bug

Change this if the connected planning system uses a different work item type for defects.

Scoring prompts

Scoring prompts provide optional guidance about the types of defects that should receive additional consideration. Prompts are entered as comma-separated values.

For example:

breaks user flow

Other examples could include:

• data loss

• authentication

• payment failure

• security impact

• customer-facing impact

These prompts are evaluated against the bug title and description and contribute to the overall defect score.

How scoring prompts influence ranking

The scoring model combines three signals:

• The overall score is calculated as:

• Total Score = 0.5 × Priority Score + 0.2 × Age Score + 0.3 × Prompt Score

• Priority is mapped from P0 through P4, while age increases as a bug remains in the backlog. The scoring-prompt component uses the agent's reasoning to determine how strongly a bug matches the configured objectives.

• If no scoring prompts are configured, the prompt weight is redistributed between priority and age.

2. Audience

The Audience section determines who receives the results of the agent.

Select the workspace members (Viewers) who should have access to the agent's outcomes. Selected members receive an email after each agent run when email delivery is configured.

Selecting users

Use the Find field to search for a workspace member by name or email. Select the required users using the checkbox next to their name.

Workspace administrators are included as audience members by default.

If there are no users with the required viewer role, the audience list will indicate that no eligible users are available.

3. Integrations

The Integrations section shows the tools required by the agent.

The integrations are grouped according to their purpose.

Coding Tools

Coding tools code context when a customer issue can be associated with an internal story and corresponding code-area information.

Code context can include information such as:

• likely fix locations

• methods and files

• similarity of potential fix locations

• function signatures

• relevant specification or documentation context

• confidence of the identified fix location

• affected code domains

This additional context can be used to refine the estimated engineering effort for selected internal issues.

The implementation does not require coding tools integration to be used for the baseline ranking. Code context is an optional refinement stage for the highest-scoring tickets.

Planning Tools

The planning integration provides the backlog bugs that the agent evaluates. The agent supports Jira/ADO-based backlogs where configured.

The planning integration is required because the agent needs access to the current backlog and the associated issue metadata.

Running Internal Defect Ranker

The agent can be run in two ways:

Scheduled run

When the Scheduler is enabled, the agent runs automatically according to the configured cadence.

Each scheduled execution evaluates the current backlog and produces a capacity-aware defect recommendation.

Manual run

A user can initiate an immediate run using Run now.

A user can initiate an immediate execution using Run now.

When Run now is selected, Cubyts displays a confirmation dialog before starting the run.

The confirmation dialog shows the estimated cost of the run, based on recent executions.

The user can choose:

• Cancel

• Continue

The displayed cost is an estimate and is shown before execution.

What happens during a run?

Once a run starts, Cubyts follows a controlled sequence.

Step 1: Get prior runs

The agent first checks previous Internal Defect Ranker runs.

This allows it to determine whether the eligible backlog has changed since the previous run.

If the same backlog state and recommendation are unchanged, the agent can avoid repeating unnecessary work.

Step 2: Identify eligible bugs

The agent retrieves open bugs from the connected planning system.

For Jira, eligible bugs are identified from the backlog and exclude closed issues and issues already associated with a sprint.

For ADO, open bugs are retrieved according to the configured work item type and status rules.

The resulting set forms the pool of defects that the agent evaluates.

Step 3: Establish baseline estimates

The agent first determines an effort estimate for every eligible bug using available estimation information.

The estimation process follows a priority order:

1. Existing planned estimate

2. Original estimate

3. Story-point-based estimate

4. Historical benchmark estimate

5. No estimate available

For Jira and ADO, the agent uses the estimation information available from the respective planning system and historical benchmarks where applicable.

This first stage is intentionally deterministic and does not use the LLM to invent an initial estimate.

Step 4: Score the bugs

Every eligible bug receives a score between 0 and 1.

The score combines:

• Priority

• Age

• Configured scoring objectives

Priority contributes the largest component of the score.

Age ensures that older backlog defects receive increasing consideration.

Scoring prompts allow the engineering team to express additional objectives such as: breaks user flow or payment failure

The result is a ranked list of bugs, with the highest-scoring bugs considered first for deeper analysis.

Step 5: Refine the estimates using code context

For the highest-ranked candidates, the agent can perform deeper code-context analysis.

The agent looks for a relationship between the backlog bug and the corresponding engineering story/code context.

The analysis considers signals such as:

The agent uses these signals to refine the expected engineering effort.

Estimate ranges

Code-context refinement can produce an estimate range rather than a single value.

Typical calibration ranges include:

• 3–5 hours — focused change with high confidence

• 5–8 hours — several files with a clear root cause

• 10–14 hours — broader or lower-confidence change

The upper end of the estimate range is used when calculating the available budget. The report displays the estimate as a range where applicable.

Step 6: Pick defects within the engineering budget

After scoring and estimate refinement, the agent determines which bugs should actually be selected. The selection uses a deterministic knapsack algorithm.

The objective is to: Maximise the combined score of selected bugs while keeping total estimated effort within the configured bug budget.

For example, with a 19-hour budget, the agent may select three defects whose combined estimate is 18 hours.

The agent does not simply select bugs in rank order. It considers combinations.

This means a slightly lower-ranked bug may be selected if including it produces a better overall use of the available capacity.

The selection is deterministic: the same input data produces the same selection.

Step 7: Generate the report

The final report contains:

• Selected defects

• Estimated effort

• Priority

• Reason for selection

• Important defects that were not selected

• Reason those defects remained outside capacity

• Engineering capacity used and remaining

The outcome is also recorded in the agent run history for auditability.

When configured, an email version of the report is also sent to the selected audience.

Understanding the Internal Defect Ranking Report

The report provides both the selected defects and the context required to understand why they were selected.

The supplied example shows:

• Defects picked: 3

• Engineering capacity used: 18h of 19h

Selected defects

The selected defects are displayed with information such as:

• Work Item

• Defect

• Reason

• Priority

• Estimate

The supplied example contains:

UDY-22 — Password reset endpoint returns the reset token in the HTTP response body

• Priority: P2

• Estimate: 4–6h

• Reason: Reset tokens in HTTP responses expose sensitive credentials and break the expected secure password-reset flow.

UDY-21 — PaymentService validation order causes wrong error message for invalid amount

• Priority: P2

• Estimate: 5–7.5h

• Reason: Incorrect validation ordering returns wrong error messages and confuses merchants entering invalid payment amounts.

UDY-19 — Revenue report showing yesterday's stale numbers — “Refresh” button not bypassing cache

• Priority: P2

• Estimate: 3–4.5h

• Reason: Stale revenue information can misguide finance decisions and hide recent refunds or payments.

The report therefore provides more than a ranking. It explains why each defect was considered a good use of the available engineering capacity.

Other Critical Bugs

The report separately displays important defects that were outside the current engineering capacity.

The supplied example identifies:

• UDY-20 — Customer charged but receipt shows “Payment Failed” — Stripe says capture went through

• UDY-23 — Password reset emails are never actually sent — the reset link comes back in the API response

• UDY-17 — Customer receives “refund processing” notification before gateway confirmation

These defects remain visible so that the engineering team can understand what was not selected and why.

A defect may remain outside capacity because:

• Its estimate is too large for the remaining budget.

• Higher-value defects consumed the available capacity.

• Another combination of defects produced a better overall score.

This distinction allows the report to separate Selected for current capacity from Important but outside current capacity.

Understanding the recommendation

Internal Defect Ranker is intended to support engineering triage decisions, not replace engineering judgement.

A defect may rank highly because of a combination such as:

High priority + long backlog age + strong scoring-objective match + manageable effort

Another defect may rank highly because it has a particularly strong match to a configured objective even if its priority is lower.

The ranking should therefore be interpreted as a capacity-aware recommendation, rather than an absolute statement that one defect is always more important than another.

Skip conditions

The agent can skip a run in situations such as:

These conditions prevent the agent from performing unnecessary analysis when there is no actionable change or no viable recommendation.

Important boundaries

The Internal Defect Ranker is an advisory and reporting agent.

It does not automatically:

• Resolve bugs.

• Change bug priorities.

• Add bugs to a sprint.

• Change engineering backlog decisions.

• Modify source code.

• Resolve dependencies.

• Automatically make sprint-planning decisions.

Where estimate write-back is enabled, the agent can write agent-generated effort estimates to the connected planning system, subject to the configured safeguards. Preview runs prevent writes.

The engineering team remains responsible for deciding which recommendations should actually be taken into the sprint.

Summary

Internal Defect Ranker gives Scrum Masters, Engineering Managers, and Engineering Leads an evidence-based view of which backlog defects deserve engineering attention within the available defect capacity.

It combines:

• Priority

• Backlog age

• Configured scoring objectives

• Baseline effort estimates

• Code-context refinement where available

• Engineering capacity

to produce a capacity-aware ranked selection of internal defects.

The result is not simply a list of high-priority bugs.

It is a decision-support report that explains:

• which defects are recommended for the current capacity,

• why they were selected,

• how much engineering effort they are expected to require,

• which important defects remain outside the available capacity, and

• why those defects were not selected.

The final decision remains with the engineering team.