Benchmarking 2.0: What We Learned After Analyzing 1.5 Million ComparablesBenchmarking 2.0: What We Learned After Analyzing 1.5 Million Comparables
A deep dive into modern transfer pricing benchmarking workflows, backed by data from 1.5M+ comparables and our global practitioner survey. See how AI is transforming comparability analysis in 2025.
Borys UlanenkoCEO of ArmsLength AI
Sven van der HeideCPO of ArmsLength AI
45min webinar
Contents↓
Contents
TL;DR - Key Takeaways
Recent court cases (Italy, Denmark, Kenya) highlight the importance of defensible benchmarking decisions and proper documentation
HMRC's new guidance (INTM485120) emphasizes quality over quantity: fewer highly comparable companies beat larger loosely comparable sets
Survey data shows around 60% of practitioners analyze loss-makers case-by-case rather than automatically excluding them
AI-powered benchmarking reduces time by 50-80% while improving consistency and creating comprehensive audit trails
The modern workflow includes automated website validation, AI-generated company descriptions, and instant report generation
Benchmarking 2.0: Lessons from 1.5 Million Comparables
Introduction
Transfer pricing benchmarking has never been more important – or more scrutinized. At the same time, most teams are still relying on processes designed 10-15 years ago.
Over the last 18 months, ArmsLength AI has analyzed 1.5 million+ comparables through our platform and conducted a global TP Benchmarking Practices Survey to understand how practitioners actually work: how they treat loss-makers, how strict their range policies are, and where AI is already changing the workflow.
In this webinar, Borys Ulanenko (CEO) and Sven van der Heide (CPO) share what they've learned and how it translates into a modern, audit-ready benchmarking workflow.
Part 1: Recent Case Law and Regulatory Developments
Global Benchmarking Disputes
Three significant court cases from the past year highlight how benchmarking practices are being scrutinized worldwide:
Italy: Convergys Case (Loss-Making Comparables)
The Italian Supreme Court (ruling No. 19512/2024, July 16, 2024) addressed a fundamental question: Should loss-making companies be automatically excluded from comparable sets?
Note
The court ruled that loss-makers should NOT be excluded automatically by default. Instead, they must be analyzed on a case-by-case basis, considering the specific business model being tested and the facts and circumstances of each case.
The tax authority had sought to raise Convergys Italy's profit markup from 5% to 7.42% by excluding all loss-making comparables from the benchmarking set. But the Supreme Court found this approach too mechanical – only companies in particular situations that clearly impair comparability (e.g., startups, companies in liquidation) may be excluded, and this requires analyzing the reasons behind the losses, not simply the fact of losses.
The ruling emphasizes that "in the normal course of business, loss-making companies do exist" and legitimate business strategies (e.g., market penetration) may result in lower profits or losses without disqualifying a company as comparable.
In the EET Group dispute (Eastern High Court, June 19, 2024), the Danish Tax Authority challenged one of Europe's largest IT distributors over transactions totaling DKK 128.8 million for 2010-2012.
When tax authorities performed their own benchmark and sought to adjust the operating margin to the median of the range, EET Group argued for adjustment to the nearest point in the interquartile range.
The court supported the taxpayer, affirming that adjustments should be made to the nearest point in the range (in this case, the third quartile) rather than automatically defaulting to the median.
Warning
While this case provided relief to the taxpayer, it's the exception rather than the rule. Standard practice globally is adjustment to the median when falling outside the range. Strong supporting arguments are essential to justify any other adjustment point.
In an extreme position, Kenyan tax authorities argued that even though the taxpayer's results were within the interquartile range, they should be adjusted to the median.
The court rejected this argument, confirming that any point within the arm's length range is acceptable – consistent with OECD guidelines. Being in the range should be sufficient; tax authorities cannot insist on targeting the median specifically.
HMRC's New Benchmarking Guidance (INTM485120)
On November 10, 2025, HMRC updated its internal manual INTM485120 with new operational guidance on "Searching for comparables." This update represents a direct challenge to the "scrape and smooth" approach that dominates much of the industry.
Key Takeaways:
1. Quality Over Quantity (The "Subset" Preference)
HMRC explicitly states that it's better to have three or four highly comparable companies than a larger set of loosely comparable ones. The guidance uses a telling example: if you have 16 companies doing generic "computer R&D" but 3 companies specifically doing software R&D matching your tested party, HMRC prefers the "perfect 3" over the "noisy 16."
Tip
However, smaller sets create risk: if one comparable is successfully challenged during an audit, the impact on your range is proportionally larger. A set of 10+ well-documented comparables provides better stability.
2. IQR is a Patch, Not a Policy
HMRC explicitly reminds case teams that neither UK law (TIOPA10) nor the OECD Guidelines mandate the use of an interquartile range. The IQR is positioned as a statistical tool to be used only when comparability defects remain and there's a sizeable number of observations. If you have a small, pristine set of comparables, the full range (min to max) might be more appropriate.
3. The "Median Trap"
If results fall outside the range, HMRC considers the median the most defensible adjustment point for Mutual Agreement Procedures (MAP). This creates a "cliff edge" risk: even a tiny miss outside the range triggers an adjustment all the way to the median, not just to the nearest edge.
4. Manual Verification is Mandatory
HMRC instructs case teams not to rely on database descriptions alone. If an HMRC officer clicks through to a comparable's website and finds they sell different products than described, that company is out – and your entire range shifts.
Part 2: Survey Insights – How Practitioners Actually Benchmark
ArmsLength AI ran the TP Benchmarking Practices Survey to capture how experienced practitioners from around the world actually approach ranges, multi-year data, loss-makers, and capital adjustments in day-to-day work.
The Core Toolkit: IQR, Three Years, and Weighted Averages
Despite regional differences, there is a remarkably stable "default toolkit" that most respondents converge on:
Interquartile range as the main range
Three years of financial data in the set
Weighted average PLIs across years
Require at least 2 out of 3 years of data to keep a company in
Multi-year benchmarking practices
Financial years used
Dataset timeframes
3 years
92.7%
1 year
5.5%
5 years
1.8%
PLI computation method
Profit level indicator calculation
Weighted average
81.8%
Simple average
18.2%
The vast majority of practitioners use 3-year datasets with weighted average PLIs, forming a de facto standard for multi-year benchmarking.
Where Practitioners Diverge: Range Types
While IQR is the shared language, it's not the only dialect. The survey reveals meaningful use of:
IQR (25th-75th percentile) – the dominant standard
Range 35-65 – particularly common in India
Full ranges (min-max) – used in specific contexts or alongside IQR
Range types used in practice
Share of respondents using each range definition.
Loading...
Percentile Methods: No Universal Standard
Percentile computation is where consensus really breaks down. Practitioners juggle multiple methods depending on jurisdiction and sample size:
Excel PERCENTILE.INC – leads globally but not by enough to be a true standard
Custom formulas – for specific regulatory requirements
Percentile methods in use
Overall share of INC, EXC, US / jurisdiction rules, and other approaches.
Loading...
Note
The US was actually the first country to prescribe interquartile range methodology in the early 1990s – before Excel's percentile functions even existed! This explains why some jurisdictions have specific calculation rules that differ from standard Excel functions.
Loss-Makers: Far More Nuanced Than "Exclude by Default"
The survey shows a profession that has moved well beyond blunt rules. Around 60% of respondents describe their policy as "case-by-case", with smaller groups in the "exclude" and "include" camps.
When you look at the actual rules people use:
Many exclude comparables that are loss-making in "most years"
A smaller group exclude only if "all years" are loss-making
Several look at multi-year average PLI rather than individual years
How practitioners treat loss-making comparables
Overall stance and the rules used when exclusion is on the table.
A. Overall stance on loss-makers
Case-by-case
60.0%
Exclude
27.3%
Include
7.3%
Other
5.5%
B. Exclusion rules when loss-makers are screened out
Most years loss-making
72.9%
All years loss-making
27.1%
Panel B is based on respondents who reported using a specific rule for excluding loss-makers (35 with "most years" vs 13 with "all years").
The picture that emerges is not "loss-makers are fine" or "loss-makers are banned", but "we use guardrails, then exercise judgment inside those guardrails."
Even with database-level independence filters, approximately 30% of companies that make it to detailed review still show independence issues based on publicly available information. Database filters alone are insufficient for independence analysis.
Live Demonstration: Distribution of Construction Materials
The webinar included a complete end-to-end benchmark demonstration for a European distribution of construction and electronic materials.
Step 1: Data Import and Website Validation
After uploading the database export (200 companies), the platform:
Validates existing website URLs
Searches for missing websites using company name and country
Identifies mismatches between database URLs and actual company websites
Flags acquired companies with outdated website information
Results: 131 of 200 websites validated, 27 alternative websites suggested where database information was outdated.
Step 2: AI Web Search and Analysis
The platform analyzes validated websites and generates:
Business descriptions: Overall company profile
Products and services: What the company offers
Functional profile: Functions and activities performed
Structure and affiliations: Independence indicators, group membership, foreign offices
Each description is grounded in website sources with screenshots captured for documentation.
Key point
Processing time: Approximately 1-2 minutes per 100 companies, with all sources and screenshots preserved for audit defense.
Step 3: Accept/Reject Analysis
Define your comparability strategy including:
Acceptable products/services
Rejectable products/services
Ideal functional profile
Functions to reject
The AI applies these criteria consistently across all companies, generating:
Accept/reject decisions
Detailed reasoning for each decision
Confidence levels (flagging when AI is uncertain)
Comparability checklists showing which criteria matched
Step 4: Human Review
Companies are prioritized for review:
High priority: All AI-accepted companies
Medium priority: Borderline cases or low-confidence decisions
Low priority: Clear rejections
Each company review includes:
AI-generated descriptions
Decision reasoning
Key financial KPIs with outlier highlighting
Website screenshots
Comparability checklist
Confidence level and doubt reasons
Tip
The system highlights financial outliers automatically. For example, if a company has 5.9% intangibles while the peer group IQR is 0-0.1%, this is flagged for human consideration – even if it may not warrant rejection.
Step 5: Export and Documentation
The platform generates:
Excel Export:
Waterfall from initial set to accepted comparables
Borys UlanenkoCEO of ArmsLength AI
Sven van der HeideCPO of ArmsLength AI