Connectome
burlington_brt · checked on the control scenario

Model validation

How closely this study's model reproduces what was observed: traffic counts, federal traffic volumes, commuting and transit ridership. Checked on the control scenario, the study's control. Published guidelines are shown for reference: they are guidelines, not pass/fail standards, and the manuals that publish them say so.

Independent check the model never saw this dataPartly independent some of this data was used to fit the modelConsistency only the model was fitted to this dataNo observed data a check on the run itself
Traffic counts0.50×modelled ÷ counted VMT vs HPMS0.66×major roads Commute time—not available Mode share—not available Transit boardings—not available Run quality1.4×10-9assignment relative gap

Traffic counts

Modelled ÷ observed volume

Independent check
0.50×

Across 26 count locations in the study area, the model carries 50 % of the counted traffic in the analysis hour.

Freeways (10)Freeways (10): 0.53×0.53×Arterials (16)Arterials (16): 0.45×0.45×1.0 = matches the counts
Technical detail

Independent check: The model never sees traffic counts: demand comes from population, jobs and travel costs, not from the counts.

By count class
Count classNGeh lt5 pctGeh lt10 pctMedian gehSum ratioRmsePct rmse
count_defect0——————
reversible_closed0——————
express_hov0——————
wrong_road0——————
out_of_area666.766.72.392.05111195
clean267.6919.221.00.5058466.8
Clean counts by road type
Road typeNGeh lt5 pctGeh lt10 pctMedian gehSum ratioRmsePct rmse
freeway10102022.20.5379852.1
smaller_arterial166.2518.820.30.4539685.5

Headline uses the clean counts; all counts together: 0.53×.

Sources
  • Traffic counts (2025-06-04 to 2025-06-25; 32 counts)

Percent RMSE by volume group

Independent check
67 %

Typical error at a single count location, as a share of the average count.

<500 veh/h (9)<500 veh/h (9): 112 %112 %500-1000 veh/h (7)500-1000 veh/h (7): 56 %56 %1000-2000 veh/h (9)1000-2000 veh/h (9): 55 %55 %2000-4000 veh/h (1)2000-4000 veh/h (1): 44 %44 %
Published guideline. FHWA (2010) Figure 9.8 compiles %RMSE targets by volume group from Florida, Ohio and Oregon: from about 100 % on links under 5,000 vehicles/day down to about 20 % above 50,000/day (values read off the published chart). Those groups are daily volumes (AADT); this model's counts are one analysis hour, so its groups are not the same links and the comparison is indicative only.
FHWA, Travel Model Validation and Reasonableness Checking Manual, 2nd ed. (FHWA-HEP-10-042, 2010), §9.1.4, Figure 9.8
Technical detail

Independent check: The model never sees traffic counts: demand comes from population, jobs and travel costs, not from the counts.

Clean counts
Volume group (veh/h)Counts%RMSERMSE (veh/h)Modelled ÷ countedMedian GEH
<50091122700.6317.3
500-1000756.23850.4920.1
1000-2000954.67670.4723.4
2000-4000143.91,3750.5627.8
>=40000————
All counts
Volume group (veh/h)Counts%RMSERMSE (veh/h)Modelled ÷ counted
<500151322210.82
500-1000756.23850.49
1000-2000954.67670.47
2000-4000143.91,3750.56
>=40000———
Sources
  • Traffic counts (2025-06-04 to 2025-06-25; 32 counts)

Modelled vs observed, by count

Independent check

32 counts: 6 out of study area, 26 clean.

10101001001,0001,00010,00010,000Counted (veh/h)Modelled (veh/h)tmas:00D059:204353209-7799470942-0 (Out of study area): counted 20, modelled 28.1tmas:00D059:7799470942-204353209-0 (Out of study area): counted 24.8, modelled 18.8tmas:00L057:204676392-592437718-0 (Out of study area): counted 134, modelled 337tmas:00L057:592437718-204676392-0 (Out of study area): counted 85.2, modelled 264tmas:00W055:204976350-6262780318-0 (Out of study area): counted 32, modelled 22.7tmas:00W055:6262780318-204976350-0 (Out of study area): counted 44.5, modelled 26.6tmas:00D040:204449629-2475715121-0 (Clean): counted 399, modelled 123tmas:00D040:2475715121-204449629-0 (Clean): counted 653, modelled 472tmas:00D061:204351663-204351683-0 (Clean): counted 482, modelled 83.6tmas:00D061:204351683-204351663-0 (Clean): counted 401, modelled 23.7tmas:00D091:204377692-204366042-0 (Clean): counted 1,678, modelled 923tmas:00D091:204497213-204372847-0 (Clean): counted 3,133, modelled 1,758tmas:00D092:204402065-204387062-0 (Clean): counted 712, modelled 633tmas:00D092:3741140672-204381100-0 (Clean): counted 1,828, modelled 1,478tmas:00D099:204390176-252157702-0 (Clean): counted 1,862, modelled 1,056tmas:00D099:204639832-252158870-0 (Clean): counted 1,384, modelled 639tmas:00D132:204573240-204606797-0 (Clean): counted 65, modelled 436tmas:00D132:204606797-204573240-0 (Clean): counted 35.8, modelled 325tmas:00D277:204616033-5337576568-0 (Clean): counted 642, modelled 218tmas:00D277:5337576568-204616033-0 (Clean): counted 648, modelled 301tmas:00D530:204390719-204425426-0 (Clean): counted 631, modelled 217tmas:00D530:204425426-204390719-0 (Clean): counted 978, modelled 414tmas:00D531:204383774-204358651-0 (Clean): counted 212, modelled 129tmas:00D531:204383774-204383845-0 (Clean): counted 238, modelled 36.0tmas:00D853:204370257-204387325-0 (Clean): counted 1,446, modelled 931tmas:00D853:204372145-204370188-0 (Clean): counted 1,014, modelled 340tmas:00D893:204562003-204590342-0 (Clean): counted 1,170, modelled 261tmas:00D893:204590342-204562003-0 (Clean): counted 526, modelled 72.3tmas:00F029:204513154-204529010-0 (Clean): counted 205, modelled 192tmas:00F029:204529010-204513154-0 (Clean): counted 132, modelled 23.9tmas:00W089:11238339453-204891587-0 (Clean): counted 1,195, modelled 204tmas:00W089:205005860-204932804-0 (Clean): counted 1,062, modelled 131
Clean countsOther classes (left out of the headlines)
Technical detail

Independent check: The model never sees traffic counts: demand comes from population, jobs and travel costs, not from the counts.

Modelled side: assigned demand + background preload (truck_and_band); two-way counts sum both directions.

Count classes (each count gets the first that applies)
Count classCountsWhat it means
Faulty count0The count's own records show a fault: one lane reads 0 in every hour, so the total misses it, or the count is doubled (every value even, or more vehicles in one lane than a lane can carry).
Reversible lane, closed0A lane closed in this direction at the count hour, such as a reversible express lane, so there is no traffic to compare.
Express / HOV lane0On or beside an express or HOV lane. Their traffic depends on tolls and occupancy rules, so they are scored apart.
Matched to the wrong road0Matched to the wrong road: more traffic than the matched road can carry, or a station whose directions matched different roads.
Out of study area6Outside the study area, where most traffic comes from outside the model.
Clean26Everything else: the fair test of the model, used in the headlines.
Counts left out of the headlines (6)
CountClassWhy it is left outCountedModelled
tmas:00D059:204353209-7799470942-0Out of study area0.6 km outside the study area2028.1
tmas:00D059:7799470942-204353209-0Out of study area0.6 km outside the study area24.818.8
tmas:00L057:204676392-592437718-0Out of study area5.7 km outside the study area134337
tmas:00L057:592437718-204676392-0Out of study area5.7 km outside the study area85.2264
tmas:00W055:204976350-6262780318-0Out of study area1.6 km outside the study area3222.7
tmas:00W055:6262780318-204976350-0Out of study area1.6 km outside the study area44.526.6
Largest differences (GEH)
CountCountedModelledGEHClass
tmas:00D091:204497213-204372847-03,1331,75827.8clean
tmas:00W089:11238339453-204891587-01,19520437.5clean
tmas:00W089:205005860-204932804-01,06213138.1clean
tmas:00D893:204562003-204590342-01,17026134.0clean
tmas:00D099:204390176-252157702-01,8621,05621.1clean
Sources
  • Traffic counts (2025-06-04 to 2025-06-25; 32 counts)

Peak-direction share at paired stations

Unavailable

This study's validation file does not include this check: it was written before the check existed. Re-running the validation step adds it.

GEH statistic

Independent checkAdvanced
8 % below 5

Share of clean count locations whose GEH is below 5.

Freeways (10)Freeways (10): 10 %10 %Arterials (16)Arterials (16): 6 %6 %
Published guideline. GEH below 5 on more than 85 % of links. TAG applies this to models whose trip tables may have been adjusted to counts (matrix estimation); a demand model without that step should not be expected to reach it.
UK DfT, TAG Unit M3.1 Highway Assignment Modelling (May 2026), §3.3.10–3.3.11, Table 2
Technical detail

Independent check: The model never sees traffic counts: demand comes from population, jobs and travel costs, not from the counts.

GEH below 10, clean
19 %
Median GEH, clean
21.0
By count class
Count classNGeh lt5 pctGeh lt10 pctMedian gehSum ratioRmsePct rmse
count_defect0——————
reversible_closed0——————
express_hov0——————
wrong_road0——————
out_of_area666.766.72.392.05111195
clean267.6919.221.00.5058466.8
Clean counts by road type
Road typeNGeh lt5 pctGeh lt10 pctMedian gehSum ratioRmsePct rmse
freeway10102022.20.5379852.1
smaller_arterial166.2518.820.30.4539685.5
Sources
  • Traffic counts (2025-06-04 to 2025-06-25; 32 counts)

TAG flow criteria

Independent checkAdvanced
12 %

Share of the 26 clean counts within TAG's flow criteria.

Published guideline. More than 85 % of links within 100 veh/h below 700 veh/h, within 15 % at 700–2,700 veh/h, and within 400 veh/h above 2,700 veh/h. Defined on hourly flows, the model's own unit. As with GEH, TAG's models may have had matrix estimation.
UK DfT, TAG Unit M3.1 Highway Assignment Modelling (May 2026), §3.3.10–3.3.11, Table 2
Technical detail

Independent check: The model never sees traffic counts: demand comes from population, jobs and travel costs, not from the counts.

All counts
22 %
Sources
  • Traffic counts (2025-06-04 to 2025-06-25; 32 counts)

Vehicle-miles travelled

Regional VMT vs HPMS

Independent check
0.66×

Modelled vehicle-km on freeways, arterials and major collectors, against federal HPMS traffic volumes.

Published guideline. Florida's guideline, quoted by FHWA (2010): regional VMT within ±5 % (acceptable) or ±2 % (preferable). Freeways, arterials and major collectors only; local roads and ramps have no HPMS counterpart.
FHWA, Travel Model Validation and Reasonableness Checking Manual, 2nd ed. (FHWA-HEP-10-042, 2010), §9.1.4
Technical detail

Independent check: HPMS traffic volumes are not used to build or calibrate travel demand. They set the background through-traffic and truck layer, which is counted on both sides.

Model
246,246 veh-km per analysis hour
HPMS
371,821 veh-km per analysis hour
Passenger demand only
0.60×

Observed side uses TMAS-measured per-cell K-factors from this study's own volumes (provenance.hour_factor). Trucks still carry the assumed truck_hour_factor. Caveat: the measurement uses whatever months observed_counts fetched — early summer on studies to date — and summer mornings carry no school traffic, so school-year K is likely somewhat higher. Direction known, size unmeasured.

Sources
  • FHWA Highway Performance Monitoring System (HPMS) (vintage 2024)
  • Hour factor (AADT → analysis hour) (measured)

VMT by road type vs HPMS

Independent check
0.56×, 0.73×

Modelled ÷ HPMS vehicle-km, by road type (freeways and expressways; arterials and major collectors), on the matched basis; the thinner bar is the netted ratio.

Freeways and expresswaysFreeways and expressways: 0.56×0.56×Freeways and expressways, netted: 0.58×0.58×Arterials and major collectorsArterials and major collectors: 0.73×0.73×Arterials and major collectors, netted: 0.75×0.75×Local roads and unclassified *Local roads and unclassified *: 0.33×0.33×Local roads and unclassified *, netted: 3.61×3.61×1.0 = matches HPMS
Matched (headline)Netted

* No HPMS counterpart for most of its length; left out of the regional ratio.

Published guideline. FHWA (2010) Table 9.2 compiles VMT guidelines by functional class from five agencies: freeways within 6–7 %; principal arterials 7–15 %; minor arterials 10–15 %; collectors 15–25 %. This comparison groups principal arterials, minor arterials and major collectors into one cell, so its guideline is the range across those classes. HPMS does not inventory local roads; that row is a coverage artifact, not a finding.
FHWA, Travel Model Validation and Reasonableness Checking Manual, 2nd ed. (FHWA-HEP-10-042, 2010), §9.1.4, Table 9.2
Technical detail

Independent check: HPMS traffic volumes are not used to build or calibrate travel demand. They set the background through-traffic and truck layer, which is counted on both sides.

Road typeModel (veh-km per analysis hour)HPMSRatioNettedGrossPassenger only
Freeways and expressways84,303146,4610.560.580.550.56
Arterials and major collectors168,844225,3590.730.750.750.75
Local roads and unclassified15,7074,3500.333.613.283.72

Observed side uses TMAS-measured per-cell K-factors from this study's own volumes (provenance.hour_factor). Trucks still carry the assumed truck_hour_factor. Caveat: the measurement uses whatever months observed_counts fetched — early summer on studies to date — and summer mornings carry no school traffic, so school-year K is likely somewhat higher. Direction known, size unmeasured.

N sections cachedN sections in areaN edges modelN edges wholly outside areaModel vmt share in areaN edges boundN edges full basisN edges pruned floorN edges eligibleBind ratePreload availableDisposition available
Coverage10,2996,38214,8282,3690.775,13823,0568,2285,7580.89yesyes
Sources
  • FHWA Highway Performance Monitoring System (HPMS) (vintage 2024)
  • Hour factor (AADT → analysis hour) (measured)

Congestion

Volume-to-capacity distribution

No observed dataAdvanced
0.0 %

Share of directional road-km at a volume-to-capacity ratio of 0.85 or more in the analysis hour. A reasonableness check: no observed data.

CollectorsCollectors: 0.0 %0.0 %FreewaysFreeways: 0.0 %0.0 %Local roadsLocal roads: 0.0 %0.0 %Multilane arterialMultilane arterial: 0.0 %0.0 %RampsRamps: 0.0 %0.0 %ArterialsArterials: 0.0 %0.0 %
Technical detail

No observed data: A reasonableness check: there is no observed side.

Road typeDirectional kmV/C ≥ 0.85V/C ≥ 1.0
collector1,3530.0000.000
freeway1780.0000.000
local2,8820.0000.000
multilane_arterial53.70.0000.000
ramp10.30.0000.000
smaller_arterial5940.0000.000
all5,0700.0000.000

directional road-km; assigned + background vehicles over directional capacity.

Trip lengths

Work trip lengths vs LODES

UnavailableAdvanced

The model's work trips by origin and destination were not saved for this study ([demand.balancing] is off).

Commute mode share

Commute mode share across the region

Unavailable

The model's work trips by mode were not saved separately for this study (its trip tables predate them). Re-running the study from the evaluation stage writes them.

Transit

Transit boardings vs NTD

Unavailable

The NTD agency match for this study has not been confirmed. `python -m connectome.cli ntd-match <study>` proposes one; the run-study pre-flight step confirms it into ntd_match.toml.

Transit passenger-miles vs NTD

Unavailable

The NTD agency match for this study has not been confirmed. `python -m connectome.cli ntd-match <study>` proposes one; the run-study pre-flight step confirms it into ntd_match.toml.

Average transit trip length

UnavailableAdvanced

The NTD agency match for this study has not been confirmed. `python -m connectome.cli ntd-match <study>` proposes one; the run-study pre-flight step confirms it into ntd_match.toml.

Commute time

Commute time

Unavailable

The model's work trips by mode were not saved separately for this study (its trip tables predate them). Re-running the study from the evaluation stage writes them.

Commute time by mode

Unavailable

The model's work trips by mode were not saved separately for this study (its trip tables predate them). Re-running the study from the evaluation stage writes them.

Commute-time distribution by mode

Unavailable

The model's work trips by mode were not saved separately for this study (its trip tables predate them). Re-running the study from the evaluation stage writes them.

Run quality

Assignment convergence

No observed data
1.4×10-9

Relative gap when the assignment stopped, after 2 iterations of tapb; target 1.0×10-5. Converged.

Published guideline. A relative gap of 10⁻⁴ (0.01 %) for link flows stable enough to compare scenarios; TAG asks for below 0.1 % on a base-year model.
Boyce, Ralevic-Dekic & Bar-Gera, "Convergence of Traffic Assignments: How Much Is Enough?", Journal of Transportation Engineering 130(1):49–55 (2004); UK DfT, TAG Unit M3.1 Highway Assignment Modelling (May 2026), Table 4
Technical detail

No observed data: a check on the run itself.

Relative gap as the assignment solver reports it: total travel time on the loaded network minus the travel time if every trip took its current shortest path, divided by the former. Zero is a perfect user equilibrium.

Demand-feedback convergence

No observed data
Converged at iteration 2

Demand and assignment re-run until travel times and the trip table stop changing.

stopping tolerance10-510-1Iteration 1: 1.30e-021Iteration 2: 1.43e-052

Change in the trip table between outer iterations (RMSE ÷ mean), log scale.

Technical detail

No observed data: a check on the run itself.

IterationAssignment gapTravel-time changeTrip-table changeTotal-trip changeCriteria met
10.0000.0000.0130.000no
20.0000.0000.0000.000yes
Time gap tol
0.025
Od rmse tol
0.005
Total trips tol
0.005
Max outer
10
Min outer
2

Noise floor

Unavailable

The noise floor was not measured for this study ([assignment.feedback] measure_noise_floor is off).

Mode-choice calibration residual

Consistency only
residual 0.013

Converged at iteration 2 of 2; tolerance 0.050. Detail on the Calibration page.

Technical detail

Consistency only: Calibration fits these targets by construction.

Coverage

No observed data
306 zones

89 of 395 zones were too sparse to model; their residents and destinations moved to a neighbouring zone, not lost. Every home-based trip was assigned a destination.

Technical detail

No observed data: a check on the run itself.

N zones original
395
N zones retained
306
N zones dropped
89
Pop before
159,312
Pop after
159,312
Snap method
nearest_zone
Generation pool period
16,095
Assigned person trips period
16,095
Unassigned person trips period
-0.000
Relative gap
0.000
Status
ok

What this was checked against

No observed data

3 observed sources.

SourceVintageFromToUsed by
Traffic counts—2025-06-042025-06-25A3, A5, A7, A8, A1, A2
FHWA Highway Performance Monitoring System (HPMS)2024——B1, B2
Hour factor (AADT → analysis hour)———B1, B2