Election Data Analysis Election Forensics Election Integrity technical

2023 VA General Election DAL File Metrics

Below you will find the current summary data and graphics from the VA 2023 General Election Daily Absentee List files. We pull the DAL file everyday and track the count of each specific ballot category in each daily file.

Note: Page may take a moment to load the graphics objects.

Linear Scale Plot:

Place your cursor over the series name in the legend at right to see the series highlighted in the graphic. Place your cursor over a specific data point to see that data points value.

Logarithmic Scale Plot:

The logarithmic plot is the same underlying data as the linear scale plot, except with a logarithmic y-scale in order to be able to compress the dynamic range and see the shape of all of the data curves in a single graphic. Place your cursor over the series name in the legend at right to see the series highlighted in the graphic. Place your cursor over a specific data point to see that data points value.

Summary Data Table:
Print  CSV  Copy  

The underlying data for the graphics above is provided in the summary data table.

Additional Data:

Additional CSV datasets stratified by Locality, City, Congressional District, State House District, State Senate District, and Precinct are available here.

Data column descriptions:
  • ISSUED” := Number of DAL file records where BALLOT_STATUS= “ISSUED”
  • NOT_ISSUED” := Number of DAL file records where BALLOT_STATUS= “NOT ISSUED”
  • DELETED” := Number of DAL file records where BALLOT_STATUS= “DELETED”
  • MARKED” := Number of DAL file records where BALLOT_STATUS= “MARKED” and APP_STATUS=”APPROVED”
  • ON_MACHINE” := Number of DAL file records where BALLOT_STATUS= “ON_MACHINE” and APP_STATUS=”APPROVED”
  • FWAB” := Number of DAL file records where BALLOT_STATUS= “FWAB” and APP_STATUS=”APPROVED”
  • MAIL_IN” := The sum of “MARKED” + “PRE_PROCESSED”
  • MILITARY” := Number of DAL file records where VOTER_TYPE= “MILITARY”
  • OVERSEAS” := Number of DAL file records where VOTER_TYPE= “OVERSEAS”
  • TEMPORARY” := Number of DAL file records where VOTER_TYPE= “TEMPORARY”
  • MILITARY_COUNTABLE” := Number of DAL file records where VOTER_TYPE= “MILITARY” and where COUNTABLE is True
  • OVERSEAS_COUNTABLE” := Number of DAL file records where VOTER_TYPE= “OVERSEAS” and where COUNTABLE is True
  • TEMPORARY_COUNTABLE” := Number of DAL file records where VOTER_TYPE= “TEMPORARY” and where COUNTABLE is True
Editorial Note:

When we first started receiving DAL data files on Sept-14-2023 after our purchase from ELECT, we noticed that there were a dozen or so records that were marked as being in a countable state, including records that should correspond to In-Person-Early votes. This is problematic as in person early voting facilities were not opened until Sept-22-2023. We respectfully raised this issue with the dept of elections and they acknowledged the error and directed registrars to correct the issues before the official start of early voting. (This can be observed in the logarithmic plot with the Overseas_Countable, On-Machine, Provisional, and Pre-Processed counts being reset to 0 on 9-21-2023.) We’d like to take a moment to thank the folks at ELECT and the registrars for listening to our concerns and correcting these errors before the start of early voting. Credit where credit is due.

All data purchased by Electoral Process Education Corp. (EPEC) from the VA Dept of Elections (ELECT). All processing performed by EPEC.

If you like the work that EPEC is doing, please support us with a donation.

Election Data Analysis Election Forensics Election Integrity Interesting mathematics technical

VA voter registrations greater than census determined eligible voters

One of our volunteers brought to our attention the following press release: They claim that “… 43 counties and independent cities in Virginia and four counties in Arizona claim to have more voters than voting-age adult citizens.”

Attempt to directly validate the claim

After reading through the press release we decided to independently try to verify the claims in the release. Note that an analysis like this has been on our list of things-to-do, but there are only so many hours in the day! The fact this press release was issued gave us a well deserved prod to complete this analysis.

EPEC has purchased the entire statewide registered voter list data from the VA Department of Elections (ELECT) and has current records as of 2023-08-01. Eligible parties can purchase data from ELECT via their website here.

The necessary data from the US Census office can be downloaded here and included the estimates of the eligible voting age citizens in each county. From the documentation on the census site, the “cvap_est” field in the census data represents “The rounded estimate of the total number of United States citizens 18 years of age or older for that geographic area and group.”

It is therefore a straightforward process to accumulate the number of registrant records in each county, as well as accumulating the number of eligible voting age citizens and compute the registration percent “REG_PCT” as (# Registered / # Eligible * 100). The below table has the results of this direct computation for each county.

The results are only slightly different than the results presented by Honest Elections Project, but still show significant issues with 38 counties being over 100%.

Adjusting for population growth since 2020 census

As the census redistricting data is circa 2020, and the eligible voter data was estimated for 2021, we can attempt to account for population shifts since the 2020 census data was collected and the voter eligibility data was computed. The US Census bureau also makes available the estimates of population growth by county year-over-year since the date of the last census here, which we can use to find the recent rates of growth or decline for each county. We can then use these rates to adjust the number of eligible voter estimates to scale with the most recent rates of population change. This is admittedly an approximation and assumes a linear relationship, but it is arguably better than taking the 2020 census and 2021 eligible voter estimates and applying them directly to the latest (2023) RVL.

The REG_PCT_ADJ column in the table below represents this adjusted estimate.

Active vs inactive registrations

An additional consideration that can be made with this data, is to attempt to consider only “Active” voter registrations vs registrations with any status assigned. Note that “Inactive” voter registrations can be immediately returned to “Active” status by simply having any type of interaction with the department of elections (or through DMV, etc), and the registrant will then be allowed to vote. Because of this easy ability to change “Inactive” records to “Active”, it is most appropriate (IMO) to include them in this analysis. However, for completeness, and in order to bound the scope of the issue, the corresponding REG_PCT_ACTIVE and REG_PCT_ADJ_ACTIVE columns have also been computed which only consider “Active” voters.


Even the most forgiving analysis we could compute with the official data from US Census and VA ELECT, which only considers active voters and attempts to adjust for population change since the census, still results in multiple (6) counties in VA having more than 100% registered voters than eligible voters, and many counties that had over 90%.

The most appropriate metric to consider, in my opinion, is the Adjusted and either Active or Inactive status results, as inactive status registrations can still be converted to active status and voted. There were 36 localities with over 100% in this category and 59 between 90% and 100%. There are 133 voting localities in total in VA.

The summary tabulated data and graphics for each of the methods of analyzing the data is presented below.

Tabulated Data Results:
GOOCHLAND COUNTY22032213081984020314.622534217111.048387096774108.453897988458107.399193548387104.889962705976
LOUDOUN COUNTY291468272527259775261988.121593707112.200173226831111.252372140753104.908863439515104.022655050994
FAIRFAX CITY17867165781589016179.2525931696112.441787287602110.431553603056104.32976714915102.464560118177
SURRY COUNTY5635544953255327.44865113427105.821596244131105.772957545076102.328638497653102.281605264085
NEW KENT COUNTY19258186861754518281.8036615372109.763465374751105.339715689632106.503277286976102.210921558649
KING WILLIAM COUNTY14025135921338013590.9343586927104.820627802691103.193788078519101.584454409567100.007840824473
MATHEWS COUNTY7353709271457099.01111761264102.911126662001103.57780651670299.2582225332499.9012381091326
NORTHAMPTON COUNTY9801930694359326.0652878146103.879173290938105.09255186971498.632750397456399.7848472298299
HANOVER COUNTY86583838008328084111.898838322103.96613832853102.937873470706100.62439961575499.6291858314583
RAPPAHANNOCK COUNTY6194595259455996.95979561651104.188393608074103.285668256898100.11774600504699.250290194552
JAMES CITY COUNTY63840604186010061066.3951247591106.222961730449104.541949577299100.52911813643998.9382128690675
FAUQUIER COUNTY56031528055296553424.3705925266105.788728405551104.87910176304199.697913716605398.840659074394
FALLS CHURCH CITY11130102531034510373.4476832119107.588206863219107.29316173264799.110681488641998.8388847479627
ACCOMACK COUNTY25327241922465024522.1840906366102.74645030426103.2819911407198.141987829614698.6535290273647
CRAIG COUNTY3917374738303807.2210828548102.271540469974102.88343951549397.832898172323898.4182404555912
CHARLES CITY COUNTY5728558457005677.65042979943100.491228070175100.88680292707597.964912280701898.3505425182942
ISLE OF WIGHT COUNTY31039296652971030281.6797400553104.473241332885102.5009189267199.848535846516397.9635220194221
SPOTSYLVANIA COUNTY105580100062100440102345.441485999105.117483074472103.16043242086599.623655913978597.7688879418126
LANCASTER COUNTY9201873690658943.53432452276101.50027578599102.87879115945696.370656370656497.6795043548532
CLARKE COUNTY12213114891161011882.4255832663105.193798449612102.78204491513398.957795004306696.6890128576076
POWHATAN COUNTY24346236242419524441.048216348100.6240958875899.611112356938897.640008266170796.6570655680737
LOUISA COUNTY29799287752900529810.5293092847102.73745905878399.961324707907599.207033270125896.5262968042565
FAIRFAX COUNTY784080721354749510747334.300776511104.612346733199104.91690254084596.243412362743796.523603861148
HENRICO COUNTY239597227831236070236137.172406976101.494048375482101.46517702306596.509933494302596.4824799406588
BEDFORD COUNTY62932605996243562871.2603534819100.796027868984100.09660955765397.05934171538496.3858520718266
NELSON COUNTY11827113841195511835.4598.929318276871699.928604320072395.223755750731996.1856118694262
FREDERICK COUNTY68222638686571566439.8543302061103.814958533059102.68234433648397.18937837632296.1290488124432
NORTHUMBERLAND COUNTY1039599041009010304.4313465051103.022794846383100.87892917570598.156590683845496.1139888942937
POQUOSON CITY9547910995009479.6573875803100.494736842105100.71039078382795.884210526315896.0899706347414
ORANGE COUNTY28633273512786528465.9231224287102.756145702494100.58693644626598.155392068903796.0833059316801
PRINCE WILLIAM COUNTY317403289258300255301330.510742882105.711145526303105.33384064477796.337446503805195.9935982874356
CHESTERFIELD COUNTY268258253920259725264714.168974025103.285398017134101.3387387005897.764943690441895.9223304835323
STAFFORD COUNTY111503103640106940108128.634023171104.266878623527103.12069601850996.914157471479395.8488016946475
AMELIA COUNTY1018099031028010370.952987928399.027237354085698.158771058449696.332684824902795.4878496848553
BOTETOURT COUNTY26424256712691027035.137004444298.193979933110497.739471398485295.395763656633294.9542071703948
GREENE COUNTY14956144321521515272.161166064398.297732500821697.929820392631694.853762734143994.4987408335423
ALBEMARLE COUNTY84004791078326583844.7847760722100.887527772774100.18989281724995.006305170239694.3493387349904
CULPEPER COUNTY37104354793720037612.219020172999.74193548387198.64879277688995.373655913978594.3283882851241
EMPORIA CITY3987371040353933.8020277481398.8104089219331101.35232967690391.945477075588694.3107958618791
KING AND QUEEN COUNTY5385515754305470.7168566286799.17127071823298.43316956671294.972375690607794.2655256184836
GLOUCESTER COUNTY30265290683058030862.894891518298.969914977109298.062738788374295.055591890124394.1842950966615
WESTMORELAND COUNTY14199136041447014454.550501815198.127159640635898.232041170820394.01520387007694.1156904069188
WARREN COUNTY30580288713036030718.0311057939100.72463768115999.550651194672995.095520421607493.9871435788555
MADISON COUNTY10409100651077010808.602150537696.648096564531196.302924791086493.454038997214593.1202745722244
FLUVANNA COUNTY20994201972141521693.887289995398.034088255895496.773804156721694.312397851972993.0999582048827
KING GEORGE COUNTY19822185771974519984.620303757100.38997214484799.186272737308894.08457837427292.9564821229435
APPOMATTOX COUNTY12256117451241512679.986583729798.719291180024296.656253688203794.603302456705692.6262809699701
ROANOKE COUNTY73141691367480074926.018335727897.782085561497397.617625525315592.42780748663192.2723528297154
SUFFOLK CITY71421654886949571088.6638879661102.77142240449100.46749522899794.234117562414692.1215794732158
HIGHLAND COUNTY1879181519201971.4056224899697.864583333333395.312703715775794.5312592.0662891134289
ARLINGTON COUNTY175053155984169220169528.629042616103.446992081314103.25866550598792.17822952369792.0104178750769
YORK COUNTY50591474655159051709.59579071698.063578212831997.836773284317992.004264392324191.7914736601402
FLOYD COUNTY11879114771242512513.126249274695.605633802816994.932311585113592.370221327967891.7196851639319
SHENANDOAH COUNTY32321309583366033764.368600682696.021984551396395.725172243696591.972667855020891.6883723375006
CUMBERLAND COUNTY7381713477507789.964933993495.238709677419494.75010558508792.051612903225891.579359604933
CAROLINE COUNTY23033218052340023813.572383924698.431623931623996.722153352969893.183760683760791.565430202818
ESSEX COUNTY8291790186108673.6448066717296.295005807200995.588419687449191.765389082462391.0920400374546
MIDDLESEX COUNTY8708830889959137.7910323059896.809338521400895.296554377458692.362423568649290.9191288203865
DINWIDDIE COUNTY20876199322184021966.364513018395.586080586080695.036208598049491.263736263736390.7387291519602
FRANKLIN CITY5877556460606148.7229330708796.98019801980295.580823269667891.815181518151890.4903353194541
CHESAPEAKE CITY176334164440181540182080.13664946697.132312438030296.844171607511290.580588300099290.3118830125735
CHARLOTTE COUNTY8391809990058968.2672279118293.181565796779693.563224497646589.938922820655290.3073001080252
CAMPBELL COUNTY40938390804350543321.159458239394.099528789794394.498855783080889.82875531548190.2099585715667
MECKLENBURG COUNTY22837219072412524290.610561056194.661139896373194.015751240989490.806217616580390.1871113734884
BATH COUNTY3353320735903556.6209933936993.398328690807894.274875119617589.331476323119890.1698552068635
HALIFAX COUNTY25042241782703026938.720303335592.64520902700792.959129899349289.4487606363389.7518505992518
WYTHE COUNTY20849201882258022508.736879432692.33392382639592.626254914600789.40655447298589.6896174500436
VIRGINIA BEACH CITY328860303604342075339791.41743699396.136812102609196.782903606145488.753635898560389.3501084547837
RUSSELL COUNTY19126185892093520829.412362669691.358968235013191.822081521020688.793885837114989.2439963083892
ALEXANDRIA CITY11121697404109245109471.650837935101.804201565289101.59342546560189.161060002746188.976460347894
SCOTT COUNTY15973155351747017465.120566055391.431024613623491.456568762798388.923869490555288.9487131866319
PITTSYLVANIA COUNTY44900432604884548732.824762855791.923431262155892.135024428591988.56587163476388.7697362312001
PAGE COUNTY17104165491878518744.748119826991.051370774554291.24689161284988.096885813148888.286062283737
CARROLL COUNTY21244206452333023390.991365966191.058722674667890.821289562399888.491213030432988.2604746288714
FRANKLIN COUNTY39895385634375043800.899836393491.188571428571491.082603665717388.14488.0415702509351
GILES COUNTY12058115491325513150.296369995290.969445492267191.693750929542187.129385137683987.8231157310733
COLONIAL HEIGHTS CITY12913119651358513641.672521681995.053367684946694.658481058508788.075082811924987.7091865457335
SOUTHAMPTON COUNTY13173126671454514460.328214226390.567205225163391.097517323570787.088346510828587.5982883122804
HOPEWELL CITY15555144901678016647.325988334492.69964243146693.438429756827986.352800953516187.0410059258396
AMHERST COUNTY22734218972503025187.877735656790.827007590890990.257703481770887.483020375549386.9346763939621
PATRICK COUNTY12896125271447514454.518055241189.091537132987989.21777917959786.542314335060486.664944151893
MANASSAS CITY23663216292505524982.937215012394.444222710037994.716645190065586.326082618239986.5750884848044
PORTSMOUTH CITY68285630857350073041.556566091192.904761904761993.487876231406585.829931972789186.3686413130011
AUGUSTA COUNTY54910530036118561409.98534289989.744218354171889.415425998549686.62744136634886.310067823732
ROCKINGHAM COUNTY56871542636224062923.041079673391.373714652956390.381836326044487.183483290488486.2370906887544
ALLEGHANY COUNTY10965103951226512110.707867412189.400733795352690.539711799216684.753363228699685.8331330736759
WASHINGTON COUNTY39282378314407544210.191846522889.12535450935988.852815062121585.833238797504385.5707664226648
WAYNESBORO CITY15564142921655016705.275269959394.042296072507693.168174414871186.35649546827885.5538132059456
HAMPTON CITY9992989709105410105555.29079341694.800303576510894.669816405103685.104828763874484.9876868565225
ROCKBRIDGE COUNTY16203156511845518417.501435575887.797344892982987.976102820884484.80628555946984.9789536042499
HENRY COUNTY36321342474065040334.796007634889.350553505535190.048800527279184.248462484624884.9068382384221
FREDERICKSBURG CITY19199175492057020786.123058542493.334953816237292.364506579353985.313563441905784.4265183583041
BUCHANAN COUNTY14607140091703016593.553194703285.772166764533288.028162676229582.260716382853884.4243534559663
CHARLOTTESVILLE CITY33751304393656536279.548327137592.304115957883293.030375394044983.246273759059283.9012650475344
MARTINSVILLE CITY9035826997809907.0411100450292.382413087934691.1977642935184.550102249488883.4658896450509
STAUNTON CITY18003168792005020246.177834951189.790523690773188.920487347104884.184538653366683.3688221925113
DICKENSON COUNTY1007194531147011364.83901241787.802964254577188.615421555876382.414995640802183.1775970576605
LUNENBURG COUNTY8036776493209346.4132699841686.223175965665285.979506446686683.304721030042983.069299160288
PULASKI COUNTY23605228212764527561.594001419885.386145776813285.644538551667482.550189907759182.8000006052786
LEE COUNTY15524149131814018075.041022619185.578831312017685.886388753271882.210584343991282.5060368125189
BRISTOL CITY12449110901354513449.919567124991.908453303802192.558174328630281.8752307124482.4540246850758
SMYTH COUNTY20124194292373523613.919425675784.786180745734285.220922614476981.858015588792982.277743265587
GRAYSON COUNTY10885104881274012750.80365296885.439560439560485.367168189953882.323390894819582.253638950504
BLAND COUNTY4624431952855258.485191778687.492904446546887.934069059078281.721854304635882.1339196077333
MANASSAS PARK CITY89388188102109983.4697342231687.541625857002989.527992150471480.195886385896282.0155739234795
NEWPORT NEWS CITY122432110842136045135707.76650213289.993752067330790.217386341021481.474512109963681.6769760913118
PETERSBURG CITY23678206262531525335.484460694793.533478174995193.457853694228381.477384949634681.411508163576
BRUNSWICK COUNTY11073107061320013152.087114337683.886363636363684.191960589501681.106060606060681.4015289507093
COVINGTON CITY3797357644354394.7592043273485.614430665163586.398362764932580.631341600901981.369645838135
WISE COUNTY24615235812921528980.998543702984.25466370015484.934961653860780.715385931884381.3671066731541
DANVILLE CITY28416260103213532023.52546252688.42694880970988.73476480049880.939785280846481.2215383045099
WINCHESTER CITY18077158671973019604.384847945991.621895590471492.208963148843980.420679168778580.9359749008522
ROANOKE CITY65501589027354072871.955561856989.068534131085189.885058655246180.095186293173880.8294487856874
NORTON CITY2600244330553036.4899476728285.106382978723485.625180547449379.967266775777480.4547369528534
SALEM CITY17727162712009020223.912203202688.237929318068787.653663751530580.990542558486880.4542654087636
RICHMOND CITY158141142368177060178454.84090649589.314921495538288.616817115575780.406641816333479.778166440773
LYNCHBURG CITY55508490296146061591.281629970490.315652456882590.123144917623779.773836641718279.6037989508931
GALAX CITY4016376547304780.4324973719884.904862579281284.009135202887679.598308668076178.7585642527071
TAZEWELL COUNTY27315253713249532303.55231295484.059086013232884.557263967054278.076627173411378.5393499581963
BUCKINGHAM COUNTY11026104361362013653.768595041380.954478707782780.754261530434576.622613803230576.4331102241624
SUSSEX COUNTY7121683691859072.8634850166577.528579205225978.486797599897274.425694066412675.3455621953234
RICHMOND COUNTY5629544272007224.6657089181178.180555555555677.913639561918775.583333333333375.3252845080764
BUENA VISTA CITY4380390052255206.0430839002383.827751196172284.132995624742774.641148325358974.9129413097024
NOTTOWAY COUNTY965192721243512415.051655544177.611580217129177.736285500593874.563731403297174.6835394427009
PRINCE EDWARD COUNTY13443127101788017855.570433119375.184563758389375.287429490717171.085011185682371.182268007663
PRINCE GEORGE COUNTY24912227513244032542.605702590876.794081381011176.551952316518770.13255240443969.9114269088438
NORFOLK CITY138210126445184395182802.31049888374.953225412836675.606265381883468.572900566718269.170351104929
MONTGOMERY COUNTY60377545937903079093.969241665476.397570542831876.335782081593169.078830823737869.02296157776
WILLIAMSBURG CITY1012787211298513178.84306220177.989988448209576.842860577389167.162110127069766.174245788033
GREENSVILLE COUNTY6395615495959438.6146161934866.649296508598267.753587364700164.13757165190265.2002465429811
HARRISONBURG CITY25205228333872038443.460777083465.095557851239765.563816291547358.969524793388459.3937162223725
RADFORD CITY917184181431514484.613106033164.065665385958863.31546402285458.805448829898758.1168439804149
LEXINGTON CITY4087341064706490.8906228978963.168469860896562.965165143629252.704791344667752.5351634792698
Election Data Analysis Election Integrity technical

Ballot Completion Rate for VA Absentee Voters

Below is an analysis of the VA statewide voter completion rate for absentee ballots compiled from the 2022 General Election Daily Absentee List (DAL) file downloaded from the VA Dept of Elections (“ELECT”) on 2022-11-15 17:46:21.

The DAL file records the transactions of all absentee ballots during the early voting period in VA elections. It includes records for both mail-in and in-person early voting transactions. It does not record the the actual values of the voted ballots, but the “fact-of” a registered voters checking in to an early voting site, or mailing their ballot application or completed ballot to the registrar, etc. The DAL record is published daily over the course of the early voting period and the file is cumulative.

For the purposes of this analysis a “Completed” ballot is a ballot that has been recorded in the DAL file as reaching a state in which the ballot can be considered to be tabulate-able. A “Completed” ballot must have its “APP_STATUS” field set to “Approved” AND have the “BALLOT_STATUS” field set to (“FWAB” OR “Marked” OR “On Machine” OR “Pre-Processed”).

The “VOTER_TYPE” field was used to separate records into “Military”, “Overseas” or “Temporary(Federal-Only Ballot)” and the ballot completion rate was computed for each sub-category, as well as overall.

All Absentee Voters
Avg Transactions Per Voter1.03
Avg Completion Rate Per Voter91.95%
Num Of Unique Voters1,057,268
Military Voters (VOTER_TYPE==”Military”)
Avg Transactions Per Voter1.08
Avg Completion Rate Per Voter78.60%
Num Of Unique Voters9,346
Overseas Voters (VOTER_TYPE==”Overseas”)
Avg Transactions Per Voter1.17
Avg Completion Rate Per Voter63.63%
Num Of Unique Voters7,052
Temporary Federal Voters (VOTER_TYPE==”Temporary(Federal-Only Ballot)”
Avg Transactions Per Voter1.21
Avg Completion Rate Per Voter61.14%
Num Of Unique Voters1,539

The data above shows that there is a distinct statistical discrepancy in the ability of Military, Overseas, or Temporary Federal Workers to complete their absentee ballots in comparison to standard ballots. These categories of voters are specifically reliant on the Mail-In absentee ballot process, and are demonstrably not having the same ability to have their votes cast and counted as is provided to standard absentee voters.

This discrepancy might be due to any number of potential reasons or mechanisms, which cannot be determined from the DAL data as provided by ELECT. The discrepancy demonstrably exists, though, and it should be investigated and remedied by legislators and officials in order to remedy the comparative disenfranchisement of specific classes of VA voters..

I will note for completeness that the first discovery and observation of this discrepancy was due to the diligent work of a fellow EPEC board member. I independently validated his results and created the scripts to process the data on a statewide basis to produce these tables. As always I am happy to provide the raw data, scripts and results to interested parties that are capable of receiving and handling VA election data according to VA law and the policies of ELECT. Interested parties can contact us to request more information.

Election Data Analysis programming technical

Changes in number of detected cloned VA voter registration records Nov 2022 – July 2023

BLUF: The number of detected exact (Full Name + DOB) “clone” registration records in the VA Registered Voter List (RVL) file has decreased overall from 2022-11-23 to 2023-07-01, however there are additional new clones still being added to the database.

There has been a concentrated effort by various election integrity groups and public officials around the state of VA to clean up the voter rolls. Specifically the VA Department of Elections (“ELECT”) made a concerted effort in early 2023 to remove and clean up a large number (~19,000) of deceased voters and other errant records. The data below shows that these efforts have made an impact on the number of exact “clones” identified in the database, but that there is still more work to do.

BACKGROUND: This is a continuation of exploratory analysis on the existence of “cloned” records in the VA Registered Voter List. Please see previous posts here, here, here and here for background information.

As a reminder and for the purposes of this analysis, a potential “cloned” record is defined as a record in the VA Registered Voter List (RVL) where the Full Name (First + Middle + Last + Suffix) and full Date of Birth (mm/dd/yyyy) exactly matches another record but they have different Voter Identification Numbers. In my previous analyses I was focusing on the Active registration records that had been identified as clones, but even if a cloned record is marked as Inactive in the database, it still holds the potential to be voted as any interaction with the voter immediately moves the registration from Inactive to Active. Therefore the analysis below includes both Active and Inactive records.

It is important to note and emphasize that this analysis is only specifically focusing on exact clones, and not any other of the number of potential errors that could be represented in a voter database. There are a couple of reasons for this narrow focus:

  1. The detection of exact clones in the database requires no additional data correlations and can operate directly on the data provided from ELECT. It is easily defined and scriptable, and can be replicated by other researchers and public officials for verification.
  2. Due to item (1), the identification of exact clones is a good candidate to track over time as a proxy indicator for issues with the database.
  3. There are some rather interesting non-random distribution patterns of the already observed cloned records that I have previously discussed here, and I am interested in observing and understanding the cause of these distribution shapes.

DETAILS: As we now have collected multiple statewide voter registration lists, I was curious as to how the numbers of detected cloned registrant records have changed over time, specifically with respect to the REGISTRATION_DATE field that is reported in each record.

In Figure 1 below I’ve plotted the number of identified cloned records in the 2022-11-23 RVL stratified by registration date year. Figure 2 is the same plot from the 2023-06-30 RVL file we recently purchased. Figure 3 shows the number of additions, removals and net change in the number of identified cloned records in each date bin between these two datasets, based on the unique set of cloned Voter ID Numbers that fall within each bin.

Figure 1 Distribution of identified cloned records with respect to registration date in the 2022-11-23 Registered Voter List purchased from VA Department of Elections. The total number of identified (First + Middle + Last + Suffix + DOB) clones in the dataset was 2,445.
Figure 2 Distribution of identified cloned records with respect to registration date in the 2023-06-30 Registered Voter List purchased from VA Department of Elections. The total number of identified (First + Middle + Last + Suffix + DOB) clones in the dataset was 1,485.
Figure 3 Differences between the 2023-06-30 and 2022-11-23 RVL datasets.

The total number of identified clones in the 2022-11-23 dataset was 2,445. The total number of identified clones in the 2023-06-30 was 1,485. We can see in Figure 3 that while there has been an overall reduction of 960 in the number of cloned records (which is good!), there are still new clones being added to the voter registration database, even as previously identified clones have been removed. This suggests that there is an ongoing process(es) or mechanism(s) that is continuously adding cloned records to the voter list database.

It is not readily apparent as to what causes the added cloned records. It could be any number of technology issues, such as a poor input verification or coding practices, or related to human error and poor procedures and/or training, or a mixture of issues.

The two full datasets above (2022-11-23 and 2023-06-30) were purchased directly from the department of elections. The lists were parsed, standardized and normalized for string case, known typos, whitespace and punctuation issues, but otherwise the raw data entries were unadjusted.

We also purchased the Monthly Update Subscription (MUS) from ELECT at the time that we ordered the 2023-06-30 RVL. The MUS is generated on a monthly basis and captures the changes to the voter list over the prior month. We received the 2022-07-01 MUS, and applied the changes within it to the full 2023-06-30 RVL that we had received the day before. As the MUS contains all the changes over the previous month, and we had purchased our full dataset the prior day, we did not expect there to be many adjustments required, but there were a few. Applying the MUS to the 2023-06-30 dataset resulted in the generation of an updated 2023-07-01 dataset.

For completeness, the same plots that were generated above for the directly purchased data is repeated below for the updated RVL dataset. Figure 4 plots the number of identified cloned records in the 2023-07-01 RVL stratified by registration date year, and Figure 5 shows the differences with the 2022-06-23 dataset.

Figure 4 Distribution of identified cloned records with respect to registration date in the 2023-07-01 Registered Voter List generated from the purchased 2023-06-30 RVL and 2023-07-01 MUS files. The total number of identified (First + Middle + Last + Suffix + DOB) clones in the dataset was 1,575.
Figure 5 Differences between the 2023-07-01 and 2022-11-23 RVL datasets.

One thing that I find interesting is the difference in detected cloned entries between the purchased 2023-06-30 dataset and the 2023-07-01 dataset after the MUS entries have been applied. This is presented in Figure 6 below. We see that the application of the MUS did not remove any clones, but 90 were added. I’m not sure what this means yet, as we only have a single MUS file and it was generated so close to our full download. We will monitor and see how this progresses as we continue to receive the MUS files throughout the 2023 election cycle.

Election Data Analysis Election Forensics Election Integrity mathematics programming technical

Voter ID Number distribution patterns in VA Registered Voter List

One thing that I have been asked about repeatedly is if there is any sort of patterns in the assignment of voter ID numbers in the VA data. Specifically, I’ve been asked repeatedly if I’ve found any similar pattern to what AuditNY has found in the NY data. It’s not something that I have looked at in depth previously due mostly to lack of time, and because VA is setup very differently than NY, so a direct comparison or attempt to replicate the AuditNY findings in VA isn’t as straightforward as one would hope.

The NY data uses a different Voter ID number for counties vs at the state level, which is the “Rosetta Stone” that was needed for the NY team to understand the algorithms that were used to assign voter ID numbers, and in turn discover some very (ahem) “interesting” patterns in the data. VA doesn’t have such a system and only uses a single voter ID number throughout the state and local jurisdictions.

Well … while my other machine is busy crunching on the string distance computations, I figured I’d take a crack at looking at the distribution of the Voter ID numbers in the VA Registered Voter List (RVL) and just see what I find.

To start with, here is a simple scatter plot of the Voter ID numbers vs the Registration date for each record in the 2023-07-01 RVL. From the zoomed out plot it is readily apparent that there must have been a change in the algorithm that was used to assign voter identification numbers sometime around 2007, which coincides nicely with the introduction of the current Virginia Election and Registration Information System (VERIS) system.

From a high level, it appears that the previous assignment algorithm broke the universe of possible ID numbers up into discrete ranges and assigned IDs within those ranges, but favoring the bottom of each range. This would be a logical explanation for the banded structure we see pre-2007. The new assignment algorithm post-2007 looks to be using a much more randomized approach. Nothing strange about that. As computing systems have gotten better and security has become more of a concern over the years there have been many systems that migrated to more randomized assignments of identification numbers.

Looking at a zoomed in block of the post-2007 “randomized” ID assignments we can see some of the normal variability that we would expect to see in the election cycles. We see that we have a high density of new assignments around November of 2016 and 2020, with a low density section of assignments correlated to the COVID-19 lockdowns. There are short periods where it looks like there were lulls in the assignment of voter ID’s, these are perhaps due to holidays or maintenance periods, or related to the legal requirements to “freeze” the voter rolls 30 days before any election (primaries, runoffs, etc). Note that VA now has same day voter registration as of the laws passed by the previous democratic super-majority that went into affect in 2022, so going forward we would likely see these “blackout periods” be significantly reduced.

We can see more clearly the banded assignment structure of the pre-2007 entries by zooming in on a smaller section of the plot, as shown below. It’s harder to make out in this banded structure, but we still see similar patterns of density changes presumably due to the natural election cycles, holidays, maintenance periods, legally required registration lockouts periods, etc. We can also see the “bucketing” of ID numbers into distinct bands, with the bias of numbers filling the lower section of each band.

All of that looks unremarkable and seems to make sense to me … however … if we zoom into the Voter ID address range of around [900,000,000 to 920,000,000] we do see something that catches my curiosity. We see the existence of the same banded structure as above between 900,000,000 and 915,000,000 AND pre-2007, but there is another band of assignments super-imposed on the entire date range of the RVL. This band does not seem to be affected by the introduction of the VERIS system (presumably), which is very interesting. There is also what looks like to be a vertical high-density band between 2007 and 2010 that extends along the entire vertical axis, but we only see it once we zoom in to the VERIS transition period.

The horizontal band that extends across all date ranges only exists in the [~915,000,000 to ~920,000,000] ID range. It trails off in density pre ~1993, but it exists throughout the full registration date range. I will note that the “Motor Voter” National Voter Registration Act (NVRA) was implemented in 1993, so perhaps these are a reserved universe block for DMV (or other externally provided) registrations? (That’s a guess, but an educated one.)

A plausible explanation I can imagine for the distinct high density band between 2007-2010 is that this might be related to how the VERIS system was implemented and brought into service, and there was some sort of update around 2010 that made correction to its internal algorithms. (But that is just a guess.) That still wouldn’t entirely explain the huge change in the density of new registrants added to the rolls.

Another, or additional, explanation might be that when VERIS came online there were a number of registrants that had their Voter ID number regenerated and/or their registration date field updated as part of the rollout of the new VERIS software. Meaning that while VERIS was coming online and handling the normal amount of new real registrations, it was also moving/updating a large number of historic registrations, which would account for the higher density as VERIS became the system of record. That seems to be a poor systems administration and design choice, in my opinion, as it makes inaccurate those moved registrant records by giving them a false registration date. However, if that was the case, and VERIS was resetting registration dates as it ingested voter records into its databases, why do we see any records with pre-2007 registration dates at all? (This is again, merely an educated guess on my part, so take with a grain of salt.)

Incorporating the identification of cloned registrations

In attempting to incorporate some of my early results on the most recent RVL data doing duplicate record identification (technically they are “cloned” records, as “duplicates” would have the same voter ID numbers. This was pointed out to me a few days ago.) on this dataset, I did a scatter plot of only those records that had an identified exact match of (FullName +DOB) to other records in the dataset, but with unique Voter ID numbers. The scatter plot of those records is shown below, and we can see that there is a distinct ~horizontal cluster of records that aligns with the 915M – 920M ID band and pre-2007. In the post-2007 block we see the cloned records do not seem to be totally randomly distributed, but have a bias towards the lower right of the graph.

Superimposing the two plots produces the following, with the red indicating the records with identified Full Name + DOB string matches.

Zooming in to take a closer look at the 915M-920M band again, gives the following:

It is curious that there seems to be an alignment of the exact Full Name + DOB matching records with the 915M-920M, pre-2007 ID band. Post-2007 the exact cloned matches have a less structured distribution throughout the data, but they do seem to cluster around the lower right.

If the cloned records were simply due to random data entry errors, etc. I would expect to see sporadic red datapoints distributed “salt-n-pepper” style throughout the entirety of the area covered by the blue data. There might be some argument to be made for there being a bias of more of the red data points to the right side of the plot, as officials have not yet had time to “catch” or “clean-up” erroneous entries, but there is little reason to have linear features, or to have a bias for lower ID numbers in the vertical axis.

I am continuing to investigate this data, but as of right now all I can tell you is that … yes, there does seem to be interesting patterns in the way Voter IDs are assigned in VA, especially with records that have already been found and flagged to be problematic (clones).

Election Data Analysis Election Forensics Election Integrity programming technical

Preliminary Results of 2023-07-01 VA RVL duplicate detections

Below are the preliminary results from performing exact (string distance of 0) duplicate record checks on the 2023-07-01 VA Registered Voter List information. Note that these are the numbers of ordered matches discovered, not the number of individual unique registrants. Each count represents two different registration records, with unique voter IDs, that match the given criteria. Match pairs are directional, in that a pair (A,B) is counted separately from the pair (B, A). Matches are grouped into this table according to the LOCALITY_NAME of the first element of the identified pair. Some pairs can have different locality information, except for in the strictest case (3rd column), so a match might be counted in one locality while its mirror is counted in the other.

The first data column of the table below is equivalent to the criteria used by the MOU between the VA Department of Elections (ELECT) and the Department of Motor Vehicles (DMV), as discussed and documented in a previous post. There were 5,290 matches for this criteria across the state.

The second data column below is based on a match of the registrants Full Name and Day+Month+Year of birth information, but NOT the registrants listed address information. There were 1,200 matches for this criteria across the state.

The third data column is the strictest match criteria and includes the Gender and Address of the registrant. There were 208 matches for this criteria across the state.

I am only considering Active registrations in the table below. I previously computed similar statistics using the previous purchased 2022-11-23 RVL, I have not yet done a comparison of the two datasets, but will do so once I complete the string distance processing on this latest set.

Row LabelsSum of Exact Same First+Last+DobSum of Exact Same First+Middle+Last+Sfx+DOBSum of Exact Same First+Middle+Last+Sfx+Gender+Address+DOB
Grand Total52901200208
Election Data Analysis Election Integrity technical

VA Voter List data standardization and normalization

EPEC has purchased and downloaded the full statewide VA Registered Voter List (RVL), the full Voter History List (VHL) and the Monthly Update Subscription (MUS) to the voter list as of 2023-06-30. These files are provided by ELECT as comma-separated-value files, but contain numerous idiosyncrasies, formatting issues and errors.

We combined the MUS information with our baseline list to create a new Statewide voter list record incorporating all of the relevant changes. As we had just downloaded our baseline list only the day before we received the MUS, there were a number of entries in the MUS that had already been incorporated into our baseline dataset, however there were a few significant deletions / adds / modifications.

The updated RVL and VHL is currently being processed using the following methods, among others:

  • The Statewide RVL and VHL are being split into smaller data files organized by LOCALITY_NAME, LOCALITY_PRECINCT_NAME, CONG_CODE_VALUE, STHOUSE_CODE_VALUE, STSENATE_CODE_VALUE, and CITY
  • The data has been standardized and normalized to remove whitespace errors, all fields have been converted to upper case, observed field name issues have been corrected, and missing fields in the VHL have been added.
    • The VHL does not contain “LOCALITY_NAME” or “PRECINCT_NAME” fields, but does reference each by code value. The missing fields have been added into the VHL after correlating with the RVL data in order to endure commonality between the datasets, and to allow for splitting into the folder structure defined above.
    • The formatting for precinct names in the RVL is inconsistent in its use of spaces and dashes between the precinct code and name. This has been standardized to be the ” – ” separator.
    • The inconsistent use of the ampersand symbol (“&”) in county names, such as “KING & QUEEN COUNTY”, has been standardized to always use the word “AND” instead.
    • etc. We will continue to update these standardizations and error checks as we discover new issues.
  • The primary and mailing addresses from the RVL have been fed as input to an NCOA processing system ( and the resultant reports have been collated for each grouping as listed above.
  • The RVL fields have also been collated against version 13 of the US Dept of Transportation’s National Address Database and the RVL entries have been augmented with the information regarding whether a match was found or not, as well as the type of match. Our best attempt has been made to match addresses to the RVL entries, but there are still inconsistencies and mis-spellings in both the NAD and RVL data that we are continuing to work to identify and improve.
    • Prior to matching to the NAD listings the RVL primary and mailing addresses are normalized and standardized according to the US Post Offices published list of common street suffix abbreviations.
    • Initial matches are attempted based on a Strict match to either the Primary or Mailing address
    • Subsequent matches use iterative relaxing of various criteria, such as ignoring the street suffix, or flipping the position of the street direction indicator. We have denoted the USDOT_MATCH_TYPE in the augmented RVL dataset to allow filtering on these different matching criteria.

EPEC is working to make this value-added data available for those entities that are authorized to handle VA election information. Interested parties may contact us for more details.

Election Data Analysis Election Forensics Election Integrity mathematics programming technical

Potential Duplicate Registrants in VA RVL by Locality

Previously I posted the computation of potential duplicate records based on string comparisons in the registered voter list. As a follow up to that article, I’ve compiled the statistics of the number of potential pairs for each locality in VA.

I tallied the number of registrant pairs with the reference match criteria defined by the MOU between ELECT and the DMV along with the two highest confidence (most stringent) match criteria that I computed. I also stratified the results by Active registrant records only or either Active or Inactive records. I also stratified by if the pairs crossed a locality boundary or not.

The table below is organized into the following computed columns, and has been sorted in decreasing order according to column 5.

  1. Exactly matching First + Last + DOB, which is equivalent to the MOU between ELECT and DMV.
  2. Exactly matching First + Middle + Last + Suffix + DOB
  3. Exactly matching First + Middle + Last + Suffix + DOB + Gender + Street Address
  4. The same as #1, but filtering for only ACTIVE voter records
  5. The same as #2, but filtering for only ACTIVE voter records
  6. The same as #3, but filtering for only ACTIVE voter records
  7. The same as #1, but filtering for only pairs that cross a locality boundary.
  8. The same as #2, but filtering for only pairs that cross a locality boundary.
  9. The same as #3, but filtering for only pairs that cross a locality boundary.
  10. The same as #4, but filtering for only pairs that cross a locality boundary.
  11. The same as #5, but filtering for only pairs that cross a locality boundary.
  12. The same as #6, but filtering for only pairs that cross a locality boundary.

LOCALITY_NAMENum Registrant RecordsPct Same First Last DobPct Same Full Name DobPct Same Full Name Dob AddressPct Same First Last Dob _ Active OnlyPct Same Full Name Dob _ Active OnlyPct Same Full Name Dob Address _ Active OnlyPct Same First Last Dob _ xLocPct Same Full Name Dob _ xLocPct Same Full Name Dob Address _ xLocPct Same First Last Dob _ Active Only _ xLocPct Same Full Name Dob _ Active Only _ xLocPct Same Full Name Dob Address _ Active Only _ xLoc
NORTON CITY26040.2304%0.2304%0.1536%0.1920%0.1920%0.1536%0.0768%0.0768%0.0000%0.0384%0.0384%0.0000%
NOTTOWAY COUNTY97040.2988%0.2061%0.0618%0.2473%0.1752%0.0618%0.2370%0.1649%0.0206%0.1855%0.1340%0.0206%
RADFORD CITY95510.4293%0.2827%0.0000%0.2827%0.1675%0.0000%0.4293%0.2827%0.0000%0.2827%0.1675%0.0000%
HIGHLAND COUNTY19030.2627%0.1576%0.1051%0.2627%0.1576%0.1051%0.1576%0.0525%0.0000%0.1576%0.0525%0.0000%
WILLIAMSBURG CITY104800.2195%0.1336%0.0000%0.2004%0.1336%0.0000%0.2004%0.1336%0.0000%0.1813%0.1336%0.0000%
LYNCHBURG CITY563190.3072%0.1829%0.0533%0.2255%0.1296%0.0533%0.1616%0.0764%0.0000%0.1190%0.0479%0.0000%
EMPORIA CITY40230.3480%0.1740%0.0000%0.2983%0.1243%0.0000%0.2486%0.0746%0.0000%0.1989%0.0249%0.0000%
SUFFOLK CITY715800.2403%0.1229%0.0754%0.2249%0.1187%0.0754%0.1229%0.0307%0.0000%0.1104%0.0265%0.0000%
FALLS CHURCH CITY112130.1784%0.1338%0.0357%0.1516%0.1159%0.0178%0.0892%0.0624%0.0000%0.0803%0.0624%0.0000%
SUSSEX COUNTY71490.2658%0.1259%0.0839%0.2238%0.1119%0.0839%0.1539%0.0140%0.0000%0.1119%0.0000%0.0000%
FRANKLIN CITY59240.2026%0.1182%0.0338%0.1857%0.1013%0.0338%0.1688%0.0844%0.0000%0.1519%0.0675%0.0000%
APPOMATTOX COUNTY121950.2542%0.1230%0.0328%0.2214%0.0902%0.0328%0.2050%0.0738%0.0000%0.1886%0.0574%0.0000%
LEE COUNTY156190.2497%0.0960%0.0128%0.2305%0.0832%0.0128%0.1473%0.0192%0.0000%0.1409%0.0192%0.0000%
ALBEMARLE COUNTY848890.1920%0.1001%0.0212%0.1590%0.0825%0.0188%0.1402%0.0554%0.0000%0.1096%0.0401%0.0000%
AMHERST COUNTY229060.1965%0.0829%0.0437%0.1790%0.0742%0.0437%0.1441%0.0393%0.0000%0.1266%0.0306%0.0000%
PRINCE EDWARD COUNTY135950.2280%0.0883%0.0000%0.1912%0.0662%0.0000%0.2133%0.0883%0.0000%0.1765%0.0662%0.0000%
STAUNTON CITY181800.1980%0.0935%0.0000%0.1595%0.0605%0.0000%0.1650%0.0605%0.0000%0.1265%0.0275%0.0000%
NELSON COUNTY118950.1765%0.0673%0.0168%0.1513%0.0588%0.0168%0.1261%0.0504%0.0000%0.1177%0.0420%0.0000%
ARLINGTON COUNTY1770920.1378%0.0683%0.0113%0.1146%0.0576%0.0102%0.0870%0.0344%0.0000%0.0683%0.0260%0.0000%
NORTHUMBERLAND COUNTY104570.1339%0.0574%0.0191%0.1243%0.0574%0.0191%0.0956%0.0191%0.0000%0.0861%0.0191%0.0000%
SOUTHAMPTON COUNTY132180.2194%0.0757%0.0000%0.1740%0.0530%0.0000%0.1589%0.0454%0.0000%0.1286%0.0227%0.0000%
HOPEWELL CITY158250.2401%0.0695%0.0253%0.2085%0.0506%0.0253%0.1390%0.0190%0.0000%0.1201%0.0126%0.0000%
LUNENBURG COUNTY80970.1853%0.0618%0.0000%0.1729%0.0494%0.0000%0.1853%0.0618%0.0000%0.1729%0.0494%0.0000%
AMELIA COUNTY101790.1375%0.0884%0.0098%0.0884%0.0491%0.0098%0.1375%0.0884%0.0098%0.0884%0.0491%0.0098%
RICHMOND CITY1610970.1707%0.0639%0.0000%0.1316%0.0490%0.0000%0.1459%0.0528%0.0000%0.1155%0.0416%0.0000%
CHARLOTTESVILLE CITY347890.1265%0.0604%0.0000%0.1064%0.0489%0.0000%0.1150%0.0489%0.0000%0.0949%0.0374%0.0000%
LEXINGTON CITY42110.2612%0.1187%0.0000%0.1900%0.0475%0.0000%0.2612%0.1187%0.0000%0.1900%0.0475%0.0000%
FAIRFAX COUNTY7877270.1143%0.0559%0.0053%0.0988%0.0474%0.0053%0.0665%0.0236%0.0000%0.0546%0.0171%0.0000%
CHARLOTTE COUNTY84740.2242%0.0708%0.0236%0.1652%0.0472%0.0236%0.2006%0.0472%0.0000%0.1416%0.0236%0.0000%
HARRISONBURG CITY264430.1777%0.0870%0.0000%0.1210%0.0454%0.0000%0.1324%0.0567%0.0000%0.0908%0.0303%0.0000%
BRUNSWICK COUNTY110980.2253%0.0631%0.0000%0.1982%0.0451%0.0000%0.2072%0.0451%0.0000%0.1802%0.0270%0.0000%
HAMPTON CITY1008070.2044%0.0764%0.0060%0.1468%0.0446%0.0040%0.1210%0.0387%0.0000%0.0972%0.0268%0.0000%
WISE COUNTY247500.1455%0.0525%0.0000%0.1333%0.0444%0.0000%0.1212%0.0364%0.0000%0.1091%0.0283%0.0000%
WYTHE COUNTY209500.1480%0.0525%0.0191%0.1289%0.0430%0.0191%0.1002%0.0143%0.0000%0.0907%0.0143%0.0000%
CHESAPEAKE CITY1780050.1258%0.0433%0.0303%0.1140%0.0410%0.0303%0.0843%0.0062%0.0000%0.0747%0.0051%0.0000%
NEWPORT NEWS CITY1247780.1354%0.0537%0.0016%0.1122%0.0409%0.0016%0.1002%0.0313%0.0000%0.0850%0.0216%0.0000%
CUMBERLAND COUNTY74160.1483%0.0539%0.0270%0.1214%0.0405%0.0270%0.1214%0.0270%0.0000%0.0944%0.0135%0.0000%
PRINCE GEORGE COUNTY249570.1643%0.0401%0.0000%0.1322%0.0401%0.0000%0.1643%0.0401%0.0000%0.1322%0.0401%0.0000%
HALIFAX COUNTY250860.1196%0.0438%0.0239%0.1156%0.0399%0.0239%0.0877%0.0120%0.0000%0.0837%0.0080%0.0000%
SMYTH COUNTY201590.1339%0.0397%0.0000%0.1290%0.0397%0.0000%0.1141%0.0198%0.0000%0.1091%0.0198%0.0000%
FAIRFAX CITY178250.1234%0.0617%0.0000%0.0954%0.0393%0.0000%0.1122%0.0617%0.0000%0.0842%0.0393%0.0000%
CAMPBELL COUNTY413180.1380%0.0508%0.0048%0.1186%0.0387%0.0048%0.1283%0.0411%0.0000%0.1089%0.0290%0.0000%
COLONIAL HEIGHTS CITY130660.0918%0.0383%0.0000%0.0918%0.0383%0.0000%0.0918%0.0383%0.0000%0.0918%0.0383%0.0000%
CHESTERFIELD COUNTY2700840.1529%0.0478%0.0067%0.1300%0.0381%0.0059%0.1107%0.0248%0.0000%0.0937%0.0196%0.0000%
PETERSBURG CITY237400.1685%0.0421%0.0000%0.1559%0.0379%0.0000%0.1601%0.0421%0.0000%0.1474%0.0379%0.0000%
SURRY COUNTY56750.1762%0.0352%0.0000%0.1410%0.0352%0.0000%0.1410%0.0000%0.0000%0.1057%0.0000%0.0000%
STAFFORD COUNTY1112610.1222%0.0440%0.0072%0.1079%0.0351%0.0072%0.1007%0.0279%0.0000%0.0881%0.0207%0.0000%
BUCHANAN COUNTY148360.0876%0.0337%0.0000%0.0876%0.0337%0.0000%0.0607%0.0067%0.0000%0.0607%0.0067%0.0000%
PORTSMOUTH CITY683810.1536%0.0409%0.0058%0.1375%0.0336%0.0058%0.1185%0.0263%0.0000%0.1024%0.0190%0.0000%
PITTSYLVANIA COUNTY453220.1677%0.0441%0.0044%0.1522%0.0331%0.0044%0.1324%0.0221%0.0000%0.1214%0.0154%0.0000%
MECKLENBURG COUNTY229960.1522%0.0478%0.0000%0.1305%0.0304%0.0000%0.1261%0.0391%0.0000%0.1131%0.0304%0.0000%
NORTHAMPTON COUNTY98770.0911%0.0304%0.0202%0.0810%0.0304%0.0202%0.0911%0.0101%0.0000%0.0810%0.0101%0.0000%
PAGE COUNTY170950.1872%0.0351%0.0000%0.1521%0.0292%0.0000%0.1521%0.0117%0.0000%0.1170%0.0058%0.0000%
ACCOMACK COUNTY254830.1216%0.0275%0.0000%0.1020%0.0275%0.0000%0.1138%0.0275%0.0000%0.0942%0.0275%0.0000%
GRAYSON COUNTY109410.1645%0.0274%0.0000%0.1554%0.0274%0.0000%0.1462%0.0274%0.0000%0.1371%0.0274%0.0000%
ALLEGHANY COUNTY110690.1355%0.0271%0.0000%0.1084%0.0271%0.0000%0.0994%0.0090%0.0000%0.0723%0.0090%0.0000%
MATHEWS COUNTY73780.0949%0.0271%0.0271%0.0678%0.0271%0.0271%0.0678%0.0000%0.0000%0.0407%0.0000%0.0000%
BEDFORD COUNTY632400.1233%0.0300%0.0063%0.1154%0.0269%0.0063%0.1012%0.0142%0.0000%0.0933%0.0111%0.0000%
HENRICO COUNTY2404360.1152%0.0299%0.0083%0.0998%0.0258%0.0083%0.0944%0.0175%0.0000%0.0807%0.0133%0.0000%
WAYNESBORO CITY155610.1735%0.0450%0.0000%0.1285%0.0257%0.0000%0.1735%0.0450%0.0000%0.1285%0.0257%0.0000%
HANOVER COUNTY870000.1092%0.0287%0.0023%0.1011%0.0253%0.0023%0.1023%0.0218%0.0000%0.0943%0.0184%0.0000%
CRAIG COUNTY39720.1007%0.0252%0.0000%0.1007%0.0252%0.0000%0.1007%0.0252%0.0000%0.1007%0.0252%0.0000%
GALAX CITY40670.1229%0.0246%0.0000%0.1229%0.0246%0.0000%0.1229%0.0246%0.0000%0.1229%0.0246%0.0000%
ORANGE COUNTY284820.1299%0.0351%0.0000%0.1194%0.0246%0.0000%0.1299%0.0351%0.0000%0.1194%0.0246%0.0000%
DANVILLE CITY288380.1040%0.0312%0.0000%0.0902%0.0243%0.0000%0.1040%0.0312%0.0000%0.0902%0.0243%0.0000%
CARROLL COUNTY211630.1040%0.0236%0.0095%0.1040%0.0236%0.0095%0.0945%0.0142%0.0000%0.0945%0.0142%0.0000%
FREDERICK COUNTY679120.1075%0.0324%0.0088%0.0883%0.0236%0.0059%0.0898%0.0206%0.0000%0.0736%0.0147%0.0000%
MANASSAS PARK CITY90180.0665%0.0222%0.0000%0.0554%0.0222%0.0000%0.0444%0.0222%0.0000%0.0333%0.0222%0.0000%
HENRY COUNTY365390.1259%0.0246%0.0000%0.1122%0.0219%0.0000%0.0931%0.0082%0.0000%0.0848%0.0055%0.0000%
BLAND COUNTY45810.1091%0.0218%0.0000%0.1091%0.0218%0.0000%0.1091%0.0218%0.0000%0.1091%0.0218%0.0000%
SPOTSYLVANIA COUNTY1053610.0987%0.0247%0.0057%0.0873%0.0218%0.0057%0.0816%0.0095%0.0000%0.0702%0.0066%0.0000%
WINCHESTER CITY183520.1035%0.0381%0.0000%0.0708%0.0218%0.0000%0.0926%0.0381%0.0000%0.0599%0.0218%0.0000%
LANCASTER COUNTY92670.0755%0.0216%0.0000%0.0755%0.0216%0.0000%0.0755%0.0216%0.0000%0.0755%0.0216%0.0000%
KING WILLIAM COUNTY139960.1286%0.0214%0.0000%0.1143%0.0214%0.0000%0.1286%0.0214%0.0000%0.1143%0.0214%0.0000%
WESTMORELAND COUNTY142330.1827%0.0211%0.0000%0.1756%0.0211%0.0000%0.1546%0.0211%0.0000%0.1475%0.0211%0.0000%
VIRGINIA BEACH CITY3319140.1118%0.0259%0.0066%0.0967%0.0208%0.0066%0.0883%0.0114%0.0000%0.0762%0.0081%0.0000%
POWHATAN COUNTY242870.1400%0.0371%0.0000%0.1153%0.0206%0.0000%0.1400%0.0371%0.0000%0.1153%0.0206%0.0000%
BOTETOURT COUNTY263110.1178%0.0190%0.0076%0.1102%0.0190%0.0076%0.1102%0.0114%0.0000%0.1026%0.0114%0.0000%
FLUVANNA COUNTY210010.1286%0.0238%0.0000%0.1190%0.0190%0.0000%0.1095%0.0238%0.0000%0.1000%0.0190%0.0000%
SCOTT COUNTY160590.1121%0.0249%0.0000%0.1059%0.0187%0.0000%0.0996%0.0125%0.0000%0.0934%0.0062%0.0000%
ALEXANDRIA CITY1122120.0820%0.0205%0.0000%0.0686%0.0178%0.0000%0.0784%0.0169%0.0000%0.0651%0.0143%0.0000%
TAZEWELL COUNTY281470.0995%0.0178%0.0142%0.0959%0.0178%0.0142%0.0853%0.0036%0.0000%0.0817%0.0036%0.0000%
RICHMOND COUNTY56490.2301%0.0354%0.0000%0.1947%0.0177%0.0000%0.1947%0.0354%0.0000%0.1593%0.0177%0.0000%
ROCKINGHAM COUNTY568170.0845%0.0246%0.0035%0.0739%0.0176%0.0000%0.0739%0.0176%0.0000%0.0669%0.0141%0.0000%
LOUISA COUNTY295670.1150%0.0271%0.0135%0.1015%0.0169%0.0135%0.1082%0.0135%0.0000%0.0947%0.0034%0.0000%
LOUDOUN COUNTY2919140.0740%0.0219%0.0041%0.0620%0.0164%0.0041%0.0651%0.0171%0.0000%0.0531%0.0116%0.0000%
RAPPAHANNOCK COUNTY62390.0962%0.0160%0.0000%0.0801%0.0160%0.0000%0.0962%0.0160%0.0000%0.0801%0.0160%0.0000%
JAMES CITY COUNTY643900.0745%0.0186%0.0000%0.0668%0.0155%0.0000%0.0621%0.0124%0.0000%0.0544%0.0093%0.0000%
PATRICK COUNTY128620.0855%0.0155%0.0000%0.0777%0.0155%0.0000%0.0855%0.0155%0.0000%0.0777%0.0155%0.0000%
PRINCE WILLIAM COUNTY3165300.0812%0.0186%0.0000%0.0663%0.0148%0.0000%0.0711%0.0142%0.0000%0.0581%0.0104%0.0000%
AUGUSTA COUNTY549930.1455%0.0218%0.0036%0.1255%0.0145%0.0036%0.1346%0.0182%0.0000%0.1146%0.0109%0.0000%
DINWIDDIE COUNTY208350.1584%0.0384%0.0048%0.1152%0.0144%0.0048%0.1488%0.0288%0.0048%0.1152%0.0144%0.0048%
GOOCHLAND COUNTY214100.1261%0.0187%0.0000%0.1121%0.0140%0.0000%0.1261%0.0187%0.0000%0.1121%0.0140%0.0000%
MONTGOMERY COUNTY619440.0936%0.0145%0.0000%0.0807%0.0129%0.0000%0.0904%0.0145%0.0000%0.0775%0.0129%0.0000%
SHENANDOAH COUNTY323040.0960%0.0155%0.0000%0.0743%0.0124%0.0000%0.0960%0.0155%0.0000%0.0743%0.0124%0.0000%
ROANOKE COUNTY734670.0953%0.0163%0.0027%0.0830%0.0123%0.0027%0.0817%0.0109%0.0000%0.0694%0.0068%0.0000%
SALEM CITY179320.0892%0.0112%0.0000%0.0781%0.0112%0.0000%0.0892%0.0112%0.0000%0.0781%0.0112%0.0000%
NEW KENT COUNTY190220.1051%0.0210%0.0000%0.0894%0.0105%0.0000%0.0946%0.0210%0.0000%0.0789%0.0105%0.0000%
WASHINGTON COUNTY394490.1014%0.0152%0.0000%0.0887%0.0101%0.0000%0.0862%0.0051%0.0000%0.0786%0.0051%0.0000%
MADISON COUNTY104070.0865%0.0192%0.0000%0.0769%0.0096%0.0000%0.0865%0.0192%0.0000%0.0769%0.0096%0.0000%
NORFOLK CITY1412360.0984%0.0092%0.0000%0.0864%0.0085%0.0000%0.0899%0.0064%0.0000%0.0793%0.0057%0.0000%
PULASKI COUNTY238250.0881%0.0126%0.0000%0.0756%0.0084%0.0000%0.0881%0.0126%0.0000%0.0756%0.0084%0.0000%
CLARKE COUNTY122690.1060%0.0163%0.0000%0.0978%0.0082%0.0000%0.1060%0.0163%0.0000%0.0978%0.0082%0.0000%
GREENE COUNTY149260.1072%0.0067%0.0000%0.1072%0.0067%0.0000%0.1072%0.0067%0.0000%0.1072%0.0067%0.0000%
GLOUCESTER COUNTY302840.0859%0.0066%0.0000%0.0859%0.0066%0.0000%0.0859%0.0066%0.0000%0.0859%0.0066%0.0000%
WARREN COUNTY305170.0885%0.0066%0.0000%0.0852%0.0066%0.0000%0.0819%0.0066%0.0000%0.0786%0.0066%0.0000%
ISLE OF WIGHT COUNTY311790.0898%0.0064%0.0000%0.0834%0.0064%0.0000%0.0898%0.0064%0.0000%0.0834%0.0064%0.0000%
ROCKBRIDGE COUNTY162660.1230%0.0123%0.0000%0.1045%0.0061%0.0000%0.1230%0.0123%0.0000%0.1045%0.0061%0.0000%
CULPEPER COUNTY371170.0943%0.0108%0.0000%0.0808%0.0054%0.0000%0.0889%0.0108%0.0000%0.0754%0.0054%0.0000%
FAUQUIER COUNTY563960.0887%0.0071%0.0000%0.0762%0.0053%0.0000%0.0887%0.0071%0.0000%0.0762%0.0053%0.0000%
FREDERICKSBURG CITY194550.0874%0.0051%0.0000%0.0720%0.0051%0.0000%0.0874%0.0051%0.0000%0.0720%0.0051%0.0000%
FRANKLIN COUNTY398660.0602%0.0050%0.0050%0.0502%0.0050%0.0050%0.0552%0.0000%0.0000%0.0452%0.0000%0.0000%
MANASSAS CITY238150.1008%0.0042%0.0000%0.0966%0.0042%0.0000%0.0840%0.0042%0.0000%0.0798%0.0042%0.0000%
YORK COUNTY508380.0925%0.0157%0.0000%0.0669%0.0039%0.0000%0.0885%0.0157%0.0000%0.0629%0.0039%0.0000%
BATH COUNTY33580.0893%0.0000%0.0000%0.0893%0.0000%0.0000%0.0893%0.0000%0.0000%0.0893%0.0000%0.0000%
BRISTOL CITY123450.0729%0.0000%0.0000%0.0567%0.0000%0.0000%0.0567%0.0000%0.0000%0.0567%0.0000%0.0000%
BUCKINGHAM COUNTY110630.1356%0.0271%0.0000%0.0904%0.0000%0.0000%0.1356%0.0271%0.0000%0.0904%0.0000%0.0000%
BUENA VISTA CITY44320.0903%0.0000%0.0000%0.0903%0.0000%0.0000%0.0903%0.0000%0.0000%0.0903%0.0000%0.0000%
CAROLINE COUNTY228940.1005%0.0087%0.0000%0.0830%0.0000%0.0000%0.1005%0.0087%0.0000%0.0830%0.0000%0.0000%
CHARLES CITY COUNTY57200.0524%0.0000%0.0000%0.0350%0.0000%0.0000%0.0524%0.0000%0.0000%0.0350%0.0000%0.0000%
COVINGTON CITY38880.1029%0.0000%0.0000%0.0772%0.0000%0.0000%0.1029%0.0000%0.0000%0.0772%0.0000%0.0000%
DICKENSON COUNTY101440.1084%0.0000%0.0000%0.0887%0.0000%0.0000%0.1084%0.0000%0.0000%0.0887%0.0000%0.0000%
ESSEX COUNTY83180.1443%0.0000%0.0000%0.1443%0.0000%0.0000%0.1443%0.0000%0.0000%0.1443%0.0000%0.0000%
FLOYD COUNTY118520.0759%0.0000%0.0000%0.0759%0.0000%0.0000%0.0759%0.0000%0.0000%0.0759%0.0000%0.0000%
GILES COUNTY120930.0413%0.0000%0.0000%0.0331%0.0000%0.0000%0.0413%0.0000%0.0000%0.0331%0.0000%0.0000%
GREENSVILLE COUNTY64350.1709%0.0155%0.0000%0.1399%0.0000%0.0000%0.1709%0.0155%0.0000%0.1399%0.0000%0.0000%
KING AND QUEEN COUNTY54030.0740%0.0000%0.0000%0.0740%0.0000%0.0000%0.0740%0.0000%0.0000%0.0740%0.0000%0.0000%
KING GEORGE COUNTY197800.1314%0.0000%0.0000%0.0910%0.0000%0.0000%0.1314%0.0000%0.0000%0.0910%0.0000%0.0000%
MARTINSVILLE CITY90700.0992%0.0000%0.0000%0.0882%0.0000%0.0000%0.0992%0.0000%0.0000%0.0882%0.0000%0.0000%
MIDDLESEX COUNTY87460.1029%0.0114%0.0000%0.0800%0.0000%0.0000%0.1029%0.0114%0.0000%0.0800%0.0000%0.0000%
POQUOSON CITY96350.0934%0.0000%0.0000%0.0934%0.0000%0.0000%0.0934%0.0000%0.0000%0.0934%0.0000%0.0000%
ROANOKE CITY660830.0817%0.0015%0.0000%0.0666%0.0000%0.0000%0.0817%0.0015%0.0000%0.0666%0.0000%0.0000%
RUSSELL COUNTY192400.1091%0.0000%0.0000%0.1040%0.0000%0.0000%0.1091%0.0000%0.0000%0.1040%0.0000%0.0000%
Election Data Analysis Election Forensics Election Integrity mathematics programming technical

Potential duplicate registrants in VA voter list

I previously documented the utilization of the Hamming string distance measure to identify candidate pairs of duplicate registrants in voter lists. While a good first attempt at quantifying the numbers of potential duplicates in the voter rolls, using a hamming distance metric is less than ideal for reasons discussed below and in the previous article. I have since been able to update the processing functions to use a more complete Levenshtein distance (LD) metric, and made some improvements to parsers and other code utilities, etc., but otherwise the the analysis followed the same procedure, and is described below.

Using the 2022-11-23 Registered Voter List (RVL) and the 2023-01-26 Voter History List (VHL) purchased from the VA Department of Elections (ELECT) I wrote up an analysis script to check for potentially duplicated registrant records in the RVL and cross reference duplicate pairings with the VHL to identify potential duplicate votes. The details are summarized below.

Please note that I will not publish voter Personally Identifiable Information (PII) on this blog. I have substituted fictitious PII information for all examples given below, and cryptographically hashed all voter information in the downloadable results file. I will make available the detailed information to those that have the authorization to receive and process voter data upon request (contact us).

Summary of Results:

As a baseline, there were 6,464 records for STATUS=’Active’ registrants that adhered to the definition of a “duplicate” when Social Security Number (SSN) is not available, as defined by the MOU between DMV and ELECT (section 7.3) of having the same First Name + Last Name + Full Date of Birth (DOB). I’ve included a copy of the MOU between the VA DMV and ELECT at the end of this article for reference. It should be noted that most records held by DMV and ELECT have a SSN associated with them (or at least they should). SSN information is not distributed as part of the data purchased from ELECT, however, so this is the appropriate standard baseline for this work.

Upgrading our definition of a potential duplicate to [First + Middle + Last + Suffix + DOB] and using a LevenshteinDistance=0 drops the number of potential duplicates to 1,982, with each identified registrant in a pair having an exactly matching string result and unique voter ID numbers.

According to my derivations and simulations that are described in detail here, we should only expect to see an average of 11 (+/- 3) potential duplicate pairs (a.k.a. “collisions”) at a distance of 0. This is over two orders of magnitude different than what we observe in the compiled results. Such a discrepancy deserves further investigation and verification.

Allowing for a single string difference by setting LevenshteinDistance<=1 increases the pool of potential duplicates to 5,568. While this relaxation of the filter does allow us to find certain issues (described below) it also increases our chances of finding false positives as well. The LD metric results should not be viewed as a final determination, but as simply a useful tool to make an initial pass through the data and find candidate matches that still require further review, verification and validation.

Increasing to LevenshteinDistance<=2 brings the number of potential duplicates up to 32,610. When we increase to LD <= 3 we get an explosion of 183,130 potential duplicates.


For every entry in the latest RVL, I performed a string distance comparison, based on Levenshtein distance, between every possible pair of strings of (FIRST NAME + MIDDLE NAME + LAST NAME + SUFFIX + FULL DOB).  For the ~6M different RVL entries, we therefore need to compute ~3.8 x 10^13 different string comparisons, and each string comparison can require upwards of 75 x 75 individual character comparisons, meaning the total number of character operations is on the order of 202.5 Quadrillion, not including logging and I/O.

A distance of 0 indicates the strings being compared are identical, a distance of 1 indicates that there a single character can be changed, inserted or removed that would convert one string into the other. A distance of 2 indicates that 2 modifications are required, etc. 

Example: The string pair of “ALISHA” –> “ALISHIA” has an LD of 1, corresponding to the addition of an “I” before the final “A”.

I aggregated all of the Levenshtein distance pairings that were less than or equal to 3 characters different in order to identify potential (key word) duplicated registrants, and additionally for each pairing looked at the voter history information for each registrant in the pair to determine if there was a potential (again … key word) for multiple ballots to be cast by the same person in any given election.  As we allow for more characters to be different, we potentially are including many more likely false positive matches, even if we are catching more true positives.

For example: At a distance of 4 the strings of “Dave Joseph Smith M 10/01/1981” and “Tony Joseph Smith M 10/01/1981” at the same address would produce a potential match, but so would “Davey Joseph Smith M 10/01/1981” and “David Josiph Smith M 10/02/1981”. The first pair is more likely to be a false positive due to twins, while the second is more likely to be due to typo’s, mistakes, or use of nicknames and might warrant further investigation. A much stronger potential match would be something like “David Josiph Smith M 10/01/1981” and “David Joseph Smith M 10/01/1981”, with a distance of 1 at the same address. In an attempt to limit false positives, I have clamped the distance checks to <= 3 in this analysis.

The Levenshtein distance measure is importantly able to identify potential insertions or deletions as well as character changes, which is an improvement over the Hamming distance measure. This is exampled by the following pairing: “David Joseph Smith M 10/01/1981” and “Dave Joseph Smith M 10/01/1981”. The change from “id” to “e” in the first name adds/subtracts a character making the rest of the characters in the remainder of the string shift position. A Levenshtein metric would correctly return a small distance of 2, whereas the hamming distance returns 27.

Note that with the official records obtained from ELECT, and in accordance with the laws of VA, I do not have access to the social security number or drivers license numbers for each registration record, which would help in identifying and discriminating potential duplicate errors vs things like twins, etc. I only have the first name, middle name, last name, suffix, month of birth, day of birth, year of birth, gender, and address information that I can work with.  I can therefore only take things so far before someone else (with investigative authority and ability to access those other fields) would need to step in and confirm and validate these findings.


The summary totals are as follows, with detailed examples.

DMV_ELECT MOU StandardLD <= 0LD <= 1LD <= 2LD <= 3
Number of Potential Duplicate Registrant Pairs7,586 (0.12%)2,472 (0.04%)6,620 (0.11%)32,610 (0.53%)183,130 (2.99%)
Number of Potential Duplicate Registrant Pairs (Active Only)6,464 (0.11%)1,982 (0.03%)5,568 (0.10%)28,884 (0.50%)164,302 (2.85%)
Number of Potential Duplicate Ballots6,3621123,57637,028236,254
Number of Potential Duplicate Ballots (Active Only)6,2281103,54236,434232,394

Examples of Types of Issues Observed:


Example #1: The following set of records has the exact match (distance = 0) of full name and full birthdate (including year), but different address and different voter ID numbers AND there was a vote cast from each of those unique voter ID’s in the 2020 General Election.  While it’s remotely possible that two individuals share the exact same name, month, day and year of birth … it is probabilistically unlikely (see here), and should warrant further scrutiny.

Voter Record A:

AMY BETH McVOTER 12/05/1970 F 12345 CITIZEN CT

Voter Record B:

AMY BETH McVOTER 12/05/1970 F 5678 McPUBLIC DR

Example #2: This set of records has a single character different (distance of 1) in their first name, but middle name, last name, birthdate and address are identical AND both records are associated with votes that were cast in the 2020, 2021, and 2022 November General Elections.  While it is possible that this is a pair of 23 year old twins (with same middle names) that live together, it at least bears looking into.

Voter Record A:


Voter Record B:


Example #3: This set of records has two characters different (distance of 2) in their birthdate, but name and address are identical AND the birth years are too close together for a child/parent relationship, AND both records are associated with votes that were cast in the 2020 and 2022 November General Elections. 

Voter Record A:


Voter Record B:


Example #4: This set of records has again a single character different (distance of 1) in the first name (but not the first letter this time) and the last name, birthdate and address are identical.  There were also multiple votes cast in the 2019 and 2022 November General from these registrants.

Voter Record A:


Voter Record B:


Example #5: This set of records has two characters different (distance of 2) in the first and middle names and the last name, birthdate, gender and address are identical.  There were also multiple votes cast in the 2021 and 2022 November General from these registrants. Again it is possible that these records represent a set of twins given the information that ELECT provides.

Voter Record A:


Voter Record B:


Example #6: The following set of records has the exact match (Distance = 0) of full name and full birthdate (including year), and same address but different voter ID numbers.  There was no duplicated votes in the same election detected between the two ID numbers.

Voter Record A:

JAMES TIBERIUS KIRK 03/22/2223 M 1701 Enterprise Bridge

Voter Record B:

JAMES TIBERIUS KIRK 03/22/2223 M 1701 Enterprise Bridge

Example #7: The following set of records has the exact match (distance = 0) of full name and full birthdate (including year), same address but different gender and voter ID numbers.  There was no duplicated votes in the same election detected between the two ID numbers.

Voter Record A:


Voter Record B:


Example #8: The following set of records has a single punctuation character different, with the same address but different voter ID numbers.  There was no duplicated votes in the same election detected between the two ID numbers.

Voter Record A:


Voter Record B:


Results Dataset:

A full version of the aggregated excel data is provided below, however all voter information (ID, first name, middle name, last name, dob, gender, address) have been removed and replaced by a one-way hash number, with randomized salt, based on the voter ID. The full file with specific voter information can be provided to parties authorized by ELECT to receive and process voter information, Election Officials, or Law Enforcement upon request.


The MOU between the VA Department of Elections (ELECT) and the VA Department of Motor Vehicles (DMV) is also provided below for reference. Section 7.3 defines the minimal standards for determining a match when no social security number is present.

Election Data Analysis mathematics programming technical

Derivation of Expected number of String Collisions in VA Registered Voter Data

Below I present the theory and derivation as to how I arrived at the expected value of 11 collisions (+/- 3) as mentioned in my posts discussing string distance analysis (here and here). I’ve tried to make the derivation below as digestible as possible, with accessible references, but it is admittedly still a very technical read. I think its important to “show my work” on the subject, though, and I present it here and am happy to take comments and criticism (contact).

Q: How much of a chance do we actually have of getting an exact (Hamming distance of 0) collision in the full name and full date of birth? Well, there is a similar and well known probability puzzle that asks how many random people do you need to approximately have a 50% chance of 2 of them sharing the same birthday (not including the year of birth). This is known as the “Birthday Problem” in probability theory, and rather surprisingly, the answer is that you only need about 23 people in your population sample to have a 50% probability that 2 of those people will share a day-of-year of birth. To quote the wikipedia article on the matter “… While it may seem surprising that only 23 individuals are required to reach a 50% probability of a shared birthday, this result is made more intuitive by considering that the birthday comparisons will be made between every possible pair of individuals. With 23 individuals, there are 23 × 22/2 = 253 pairs to consider, far more than half the number of days in a year.” The same mathematics of the birthday problem is the basis of the Birthday Attack cryptographic exploit, and it is therefore a well-studied problem in cryptography and cyber security.

Figure 1: The computed probability of at least two people sharing a birthday versus the number of people. A recreation of the classic “Birthday Problem”.

Now, as interesting as the toy birthday problem is as described above, it is over simplified for the problem we are looking at here. Firstly, the problem setup assumes independent and identically distributed random variable (e.g. an “IID” set of variables). While this is not exactly the case, the IID assumption provides for a computable first order estimate, and in the case of the classical birthday problem the estimate has been shown to be fairly accurate under experimental conditions.

Secondly, when we start additionally considering the year of birth, or sharing of first names, middle names and last names, things get much more complicated to compute, but the method is the same. We want to determine the probability of 2 people sharing the same First Name, Middle Name, Last Name, Suffix, Month-of-Birth, Day-of-Birth and Year-of-Birth in the population of unique registrants in the Registered Voter List. This means that in addition to the 365 day-of-birth possibilities, we need to estimate the number of possible years to cover, the number of possible first names, the number of possible middle names, the number of possible last names, the number of possible suffix strings and then include these possibilities into the same formulation as the birthday problem setup.

For determining how many years we should cover, I will simply use the average life expectancy of approximately 79 years. We can therefore update our N value of the birthday problem from 365 to 365 * 79 = 28835. When we perform the same analysis as the standard birthday problem with just this new parameter included, we end up needing 200 people in our sample population to have a 50% probability of of 2 people having a match.

Figure 2: The computed probability of at least two people sharing a birthday versus the number of people in the sample population. A recreation of the classic “Birthday Problem”, but we’ve updated the analysis to include the year of birth, and assumed the average life expectancy of 79 years. This moves the 50% crossover point to a population size of 200 from 23 for the standard Birthday Problem setup.

A similar analysis can be done with the number of names being considered, etc. For each (assumed independent and uniform) variable we add to the setup, we multiply the number of possible states (N) by the number of unique variable settings.

We can estimate the universe of possible names using the frequentist method from the RVL data itself: We know that we have 6,127,859 unique voter ID’s in the RVL, and there are 14 unique SUFFIX entries, 291,368 unique FIRST names, 405,591 unique MIDDLE names, and 465,185 unique LAST names. So multiplying out 365 x 79 x 14 x 291368 x 405591 x 465185 = 2.22 x 10^22 potential states to consider.

Now unfortunately, as we start dealing with bigger and bigger N values the ability of computers to maintain the necessary precision to carry out the mathematics for direct computation becomes harder and harder, eventually resulting in Infinite or divide-by-zero answers as the probabilities get smaller and smaller. So lets begin by first determining if we can find the 50% crossover point for the unique voter ID population size. We find that we only need 410 unique First, Middle, and Last names (each) to break the 50% probability limit.

Figure 3: The computed probability of at least two people sharing a first name, middle name, last name, suffix, month-of-birth, day-of-birth, year-of-birth versus the number of people in the sample population. This assumes the Nyears = 79, Nsuffix = 14, Nfirst = 410, Nmiddle = 410, Nlast = 410.

As we increase the number of unique (first, middle, last) names under consideration, we find that we very quickly reduce the probability to near zero (again … this is assuming an IID set of variables … more on that later). In fact we only need to assume that there are 1300 unique first names, middle names and last names before the probability drops to under 1%. This is two full orders of magnitude below the actual number of unique first names, middle names and last names (each) that we find by simple examination of the RVL file, so the actual probability of a collision under these conditions should be much, much, much lower. While not exactly zero, it is computationally indistinguishable from zero given machine precision. Note (again) that this formulation is still simplified in that it assumes a uniform distribution within the N possible states, but it still serves to give a first order approximation and sanity check.

Figure 4: The computed probability of at least two people sharing a first name, middle name, last name, suffix, month-of-birth, day-of-birth, year-of-birth versus the number of people in the sample population. This assumes the Nyears = 79, Nsuffix = 14, Nfirst = 1300, Nmiddle = 1300, Nlast = 1300.

As we start approaching the limit of computational precision we have to resort to approximation methods for computing the very small, but non-zero probability of collision given the actual number of unique first, middle and last names observed in the RVL dataset. We can use the Taylor series expansion for small powers in order to do this, and our equation for computing the probability becomes: Pb = 1 – exp(-k*(k-1) / (2 *N)).

Replicating our earlier example in Figure 4 above with Nfirst == Nmiddle == Nlast == 1300 to show the comparison of the Taylor expansion to the explicit computation produces the graphic in Figure 5 below. We see that the small value approximation is close, but slightly over-estimates the directly computed probability for IID variables.

Figure 5: The computed probability of at least two people sharing a first name, middle name, last name, suffix, month-of-birth, day-of-birth, year-of-birth versus the number of people in the sample population. This assumes the Nyears = 79, Nsuffix = 14, Nfirst = 1300, Nmiddle = 1300, Nlast = 1300.

When we perform this Taylor series approximation and look to find the number of records required in order to obtain a 50% probability that any 2 records would match given our updated universe of possible matches, we end up with requiring K = 176,000,000,000, or 176 Billion records. When we again try to evaluate the Taylor series for the explicit number of unique Voter ID’s present in the RVL file, which is just over 6M, we again obtain a number that is computationally indistinguishable from 0. (To be absolutely meticulous … its a bigger number that is indistinguishable from 0 than we previously computed, but it is still indistinguishable from zero.)

Figure 5: The computed probability of at least two people sharing a first name, middle name, last name, suffix, month-of-birth, day-of-birth, year-of-birth versus the number of people in the sample population. This assumes the Nyears = 79, Nsuffix = 14, Nfirst = 291368, Nmiddle = 405591, Nlast = 465185, and the computed 50% crossover point occurs at approximately 176 Billion samples required.

Another Implementation note: In order to explicitly code the above direct computations we also need to do some clever tricks with logarithms in order to avoid numerical overflow / underflow issues as much as possible. The formula for computing the permutations, which is N! / (N-K)! = N x (N-1) x … x (N-K+1) can have numerical issues when N becomes large. However if we take the base-10 logarithm of the equation, we can use the product and quotient rules of logarithms to compute the result and avoid numerical overflow: log10( N! / (N-K)! ) = log10(N!) – log((N-K)!) = log10(N) + log10(N-1) + … + … log10(N-K+1), which is a much more stable computation.

We can perform a similar trick in order to compute the denominator of N^k by using the power property of logarithms such that log10( N^k ) = k x log10(N).

You must of course remember to reverse the logarithm once you’ve computed the log-sums. So the final computation of Pb becomes the following:

Vnr = log10(N) + log10(N-1) + … + … log10(N-K+1), where N is the number of possible states N = 365 x Nyears x Nfirst x Nmiddle x Nlast x Nsuffix.

Vt = k x log10(N)

Pa_log10 = Vnr – Vt = log10(Pa) = log10(Vnr/Vt)

Pb = 1 – 10^(Pa)

Updating from uniform distributions to non-uniform distributions

So what happens when we take into account the fact that names and birthdays are not uniformly distributed? (e.g. the last name of “Smith” is more frequent than “Sandeval”) This fact increases the probability of a collision occurring in the dataset. This increase also makes intuitive sense as we can anecdotally observe that coincident names and birthdates, while still rare … do actually happen in real life with common names.

However, in the non-uniform case we don’t have as nearly of a nice closed set of formulas for computing the probability. What we can do instead to estimate the probability is perform a number of Monte Carlo simulations of selecting K values from the weighted possibilities, and determine how many collisions occurred in each simulation trial. By setting K equal to the number of unique Voter ID values in the RVL dataset, we can directly answer the question via simulation of “how many collisions of First+Middle+Last+Suffix+DOB should we expect when looking at the VA Registered Voter List file“?

We can determine the weightings for each variable easily enough from the distributions of unique values in the data itself.

The below MATLAB weightedCollisionSim(…) function is a program that can be used to perform this analysis. It assumes that the RVL table object is a global variable to setup the trials, and uses the MATLAB built-in randsample(…) function to perform each draw.

After 100 simulation runs, the results are that for the K=6,127,859 unique voter ID’s in the RVL, we should expect to have an average of about 11 collisions at Hamming distance of 0, with a standard deviation of roughly 3.

I will note that as a validation and verification step, the MATLAB simulation code below, when used with uniform sampling, produces similar results to what we analytically derived above.

function [p,m,s] = weightedCollisionSim(k,ntrials,varargin)
% To compute the probability the ntrials must be >> 1:
% [p,m,s] = weightedCollisionSim(k,ntrials,values1,weights1,...,values2,weights2)
% [p,m,s] = weightedCollisionSim(k,ntrials,Nvalues1,weights1,...,Nvalues2,weights2)
% p = Probability of a collision
% m = mean number of collisions
% s = standard deviation of collisions

if nargin == 0
    global rvl; % Assume the RVL is an available global var

    ntrials = 100; % Number of trials
    % Population size set as num of unique voter IDs in RVL
    npop = numel(unique(rvl.IDENTIFICATION_NUMBER));

    % Convert the DOB strings to datetime objects
    dob = datetime(rvl.DOB);

    % How many unique days of the year are there?
    [ud,uda,udb] = unique(day(dob,'dayofyear'));
    % How often do they occur?
    nud = accumarray(udb,1,[numel(ud),1]);
    Ndays = numel(ud);

    % How many unique years of birth are there?
    [uy,uya,uyb] = unique(year(dob));
    % How often do they occur?
    nuy = accumarray(uyb,1,[numel(uy),1]);
    Nyears = numel(uy);

    % How many unique suffix strings are there?
    [us,usa,usb] = unique(rvl.SUFFIX);
    % How often do they occur?
    nus = accumarray(usb,1,[numel(us),1]);
    Nsuffix = numel(us);

    % How many unique first names are there?
    [uf,ufa,ufb] = unique(rvl.FIRST_NAME);
    % How often do they occur?
    nuf = accumarray(ufb,1,[numel(uf),1]);
    Nfirst = numel(uf);

    % How many unique middle names are there?
    [um,uma,umb] = unique(rvl.MIDDLE_NAME);
    % How often do they occur?
    num = accumarray(umb,1,[numel(um),1])
    Nmiddle = numel(um);

    % How many unique last names are there?
    [ul,ula,ulb] = unique(rvl.LAST_NAME);
    % How often do they occur?
    nul = accumarray(ulb,1,[numel(ul),1]);
    Nlast = numel(ul);
    % Initializing the weighting vectors
    w0 = nus;
    w1 = nud;
    w2 = nuy;
    w3 = nuf;
    w4 = num;
    w5 = nul;

    % Recursively compute results and return
    [p,m,s] = weightedCollisionSim(npop,ntrials,1:Nsuffix,w0,1:Ndays,w1,1:Nyears,w2,...

if nargin < 2 || isempty(ntrials)
    ntrials = 1;

nc = zeros(ntrials,1);
for j = 1:ntrials
    fprintf('Trial %d\n',j);
    y = zeros(k,numel(varargin)/2);
    m = 1;
    for i = 1:2:numel(varargin)
        w = varargin{i+1};
        v = varargin{i};
        if ~isempty(w) && isvector(w)
            % Non-uniform weightings
            y(:,m) = randsample(v,k,true,w);
            % Uniform sampling
            y(:,m) = randsample(v,k,true);
        m = m+1;
    [u,~,ib] = unique(y,'rows');
    nu = accumarray(ib,1,[size(u,1),1]);
    nc(j) = sum(nu > 1);
p = mean(nc>0);
m = mean(nc);
s = std(nc);