The Property Price Register — A Rant
Recently I was buying a house in Cork. I wanted to see prices of specific houses and for the area I was buying in. I had heard about the property price register so decided to check it out. The Property Price Register website seemed odd, quite difficult to filter and find properties.
Anyway I found what I needed from it at the time and eventually bought a property. A couple of months down the line I decided to go and search for my own house on it to see if the price was there. I entered the address but couldn’t find it. Turns out it was entered with an address I would not have expected, Browningstown instead of Ballinlough.
I decided to go and make a better experience myself since the site left a lot to be desired. They have various disclaimers about the register:
is not intended as a "Property Price Index"
...
The Authority acknowledges that there may be errors in the data in the Register
...
If the data filed contained typographical errors then those errors will appear on the Register.Where errors are reported to the Authority they will be brought to the attention of the Revenue Commissioners.
...
Anecdotally, I know many many people use it as a price index as there is no other resource available. People are making the biggest purchase of their life and are basing decisions partly on this data…I thought to myself, how bad could it be? Well, buckle up…
The data
The first step I made was to request API access. For those of you who aren’t developers, this is a way of publicly accessing data in a documented and structured fashion. Alas there is currently no API access. I was directed to the site to manually download a CSV file. Fine, I can work with that. Opening the file I noticed that there are ~300k entries, starting from 2010 up to November 2018. Not too bad.
Looking at the columns you can see the kind of information it contains:
Date of Sale (dd/mm/yyyy)
Address
Postal Code
County
Price (�)
Not Full Market Price
VAT Exclusive
Description of Property
Property Size DescriptionThere is a common phrase used in computer science/data processing:
bad data is worse than no data
Inspecting the data I realised how apt the phrase was. It seems that the data has all been entered manually by many different people and zero standards have been imposed. I have been trying to parse this data so I can place properties on a map and then start to come up with a useful website on the back of it. It fails to parse logically in all sorts of hilarious and unexpected ways.
Below are some of the more interesting findings:
- Terrible Formatting. Addresses have no standard format. Whoever enters the data is free to do whatever they want. Some entries are capitalised, some aren’t. The example below shows in a single day there were multiple ways of entering in apartments and listing numbers of houses. This is a very very small sample of bad data.
"21/01/2010","Apartment 72 Royal Canal Court, Royal Canal Way, Ashtown""21/01/2010","Apt 29 Waltrim Grove, Killarney Road, Bray""21/01/2010","Apt. 49 Hunters Walk, Hunterswood, Ballycullen Road""21/01/2010","No 42 Royal Canal Court, Ashtown, Dublin 15""21/01/2010","No. 11 St Jude's Terrace, St. Patrick's Road""22/01/2010","1 Barnwell Way, Barnwell Hansfield, Dublin 15"
- General Area Addresses. Related to the point above but there are often entries where the address is just the name of an area. There isn’t any house name so plotting on a map means you would see a bunch of sales right in the centre of an area. There are a LOT of these types of addresses.
Taking one example:
"25/01/2010","Poulroe, Tubber","","Clare"This area is seemingly in the middle of the countryside without a house in sight. https://goo.gl/kQ3S6j
- Duplicate data. For many properties there are multiple entries in the PPR in or around the same day. I found one property, and there could certainly be more, where someone entered in the price incorrectly and instead of correcting it, it was just entered in again with the correct price:
"16/08/2010","39 Ellensborough Drive, Kiltipper, Tallaght","","Dublin","�2,610,000.00"
"16/08/2010","39 Ellensborough Drive, Kiltipper, Tallaght","","Dublin","�261,000.00"Another fun one: a property in Dublin went up by €42,391,002 in the space of 7 months, that’s a hell of a renovation!
"14/12/2017","UNIT NO 121 FIFTH FLOOR, NORTHBANK, CASTLEFORBES ROAD","Dublin 1","Dublin","�215,311.00"
"27/07/2018","UNIT NO 121 FIFTH FLOOR, NORTHBANK, CASTLEFORBES ROAD","","Dublin","�42,606,313.37"- Lack of useful information. The property price register is unfit for purpose not just for the data it includes but also the data is does not. Property size is not adequately measured, some properties may include a little blurb ‘greater than or equal to 38 sq metres and less than 125 sq metres’ which is super vague and doesn’t really give an accurate indication of the size of the property. Other information that could be recorded: type of house (semi-d, detached, cottage, bungalow), number of bedrooms, house size (sq metres). Of course some of this info will go out of date with extensions and building works but it is certainly better than the current situation.
- Residential & commercial. In the PPR there is a mix of commercial and residential listings in the CSV distributed. There is a rough distinction between the two, commercial properties are listed as having a property description of (more on these funny looking question marks in diamonds later)
Teach/�ras�n C�naithe Ath�imhe
whereas residential properties are:
Second-Hand Dwelling house /Apartment or New Dwelling house /Apartment.
There really should be a better separation between the different types of property, or at least a nicer way to filter than to search for this obscure piece of text. The property price register site itself seems to have done this filtering. - Errors in the data. The data contains so many errors. Addresses are incorrect, formatting is a mess, prices entered are incorrect and there are spelling mistakes. Some addresses include the county in the address field, some don’t. A fun example which should Model Farm Road in Cork:
the greenway, mofel farm roadThe PPR says that errors can be reported to be fixed. It would take months to trawl through the data to correct all the errors. Besides, new errors would come in daily while previous entries are fixed. It’s a futile battle, it should be impossible to enter an invalid address. They should remove the possibility of human error.
- Multiple apartment & property sales. The disclaimer on the PPR site does indicate that there are potential sales with multiple sites or properties on it and indeed there are. There are often scenarios where a developer may buy several properties for a single sum and so they cannot be split effectively. However I think these should be categorised differently, perhaps as commercial sales until they are again released back into the market.
- Missing Eircodes. It is completely baffling that a government organisation does not use their own Eircode system. It would have fixed so many of the data entry issues as it would matter less what was entered if the Eircode is there. It further shows what a shambles Eircode is when not adopted by the government itself.
- Inconsistent use of Irish. It seems that there are a few gaelgoirs that have been adding into the PPR (see below, also extra points for inconsistent address entry). Several properties are entered entirely in Irish for areas that are not predominantly Irish speaking (e.g. Dromcondra). Of course there is nothing wrong with Irish being used, it is an official language in Ireland after all. But you cannot mix it in with English. The data should be separated out into Irish and English entries. The mixture makes searching for addresses difficult, how does someone know whether they should search in Irish? It just makes the whole data set more of a mess.
26 CEIDE BHREANAINN, AN CHULOG, BAILE �THA CLIATH 5
87 B�thar an Bhreatnaigh, Droim Conrach, B.�.C 3- Special characters. Browsing the register you quickly see an abundance of funny looking � characters. In software development this character is synonymous with an unknown character. What this means is that the original data most probably does have the correct character, but when it was exported it got lost. Presumably they are exporting to ASCII instead of a UTF8 format. The repercussions for this issue are pretty wide-reaching however. All addresses with fadas are littered with these little diamonds and a reader needs to use their knowledge of the Irish language to try and fill in the gaps. Think of places and properties with mór, dún, bóthar, bán, slí, páirc in their address, these will be riddled with little diamonds. Some of these can be rectified and guessed, others cannot be unless you google the name or have local knowledge of the area. It’s not always possible to use geocoding (the process of asking google maps if it knows the address) to get you out of jail here.
- CSV format. CSV is an inappropriate format to distribute the PPR, since there are commas in some of the fields. It is also going to become slow and unwieldy as years progress and number of entries grows, it already struggles to open on my moderately spec’d machine.
- Apartments over restaurants. This is my favourite but also a really odd one. There seems to be a penchant for entering sales for apartments over restaurants by not referring to the address at all but using a colloquial phase (over restaurant) address. See examples below:
APT. OVER GIOVANNI’S TAKEAWAY, MAIN STREET, GALWAY
APARTMENT OVER RESTAURANT, BRIDGE STREET, CARRICK-ON-SHANNON
APT 1 OVERHEAD MAMMA MIA RESTAURANT, MAIN STREET, MAYNOOTH
APARTMENT OVER RESTAURENT, O’GROWNEY STREET, ATHBOY
APT. OVER CHINESE RESTAURANT, MAIN STREET, WHITEGATE
APT OVER CLAYMORE RESTAURANT, NORTHGATE STREET, ATHENRY
FLAT 1 OVER GOLDEN SEA RESTAURANT, THOMAS ST, CASTLEBAR
OVERHEAD APARTMENT, THE CHINESE RESTAURANT, PARNELL STREET MOUNTMELLICK
APT OVER LANG’S RESTAURANT, GRANGE, SLIGOA postman would find these addresses because of the local knowledge but good luck to a computer trying to figure it out.
What next for the PPR?
A number of enterprising folk have spent time trying to clean and map the data with varying degrees of success. The mapping is typically out of date because it needs to be manually done, again showing how poor the data is.
https://www.shanelynn.ie/the-irish-property-price-register-geocoded-to-small-areas/
The question really should be — why do people have to go to this effort????
The PPR is not a ‘price index’. Thats fine but in that case we have to ask why isn’t there one for Ireland? It is ridiculous that it doesn’t exist. I would love to build one but it is useless unless the government add to this database — it cannot be done by a third party.
It is time for the government to clean up this mess and come up with a proper Property Price Index system. My suggestions would be:
- Clean up the current data. It can serve as a starting point for a new system. It’ll be far from perfect but this can be a known limitation going forward.
- Have a public API so developers can integrate it easily.
- Make it impossible to enter in data in an unstructured fashion.
- Human review of new entries to ensure that data quality is maintained.
- Evaluate what extra data is required to make this system fit for purpose.
- Make Eircodes and latitude and longitude coordinates mandatory.
