Big Data and Civil Rights and Questions about Data’s Intended Uses

Big data is all the rage right now and it is going to affect virtually everything we do on and off-line.

It is illegal for me to as a Realtor to tell my clients (or anybody) that all the “x race live here;” but would it be illegal for one of the new “lifestyle real estate” search engines to make the same recommendation?

Think about it.

You decide what data is about the moment you define its schema.

With the new, data-is-abundant model, we collect first and ask questions later. The schema comes after the collection. Indeed, big data success stories like Splunk, Palantir, and others are prized because of their ability to make sense of content well after it’s been collected –  sometimes called a schema-less query. This means we collect information long before we decide what it’s for.

And this is a dangerous thing.

When bank managers tried to restrict loans to residents of certain areas (known as redlining) Congress stepped in to stop it (with the Fair Housing Act of 1968). They were able to legislate against discrimination, making it illegal to change loan policy based on someone’s race.

“Personalization” is another word for discrimination. We’re not discriminating if we tailor things to you based on what we know about you,  right? That’s just better service. O’Reilly Radar (no longer available)

(thanks Boing Boing for pointing out that article)

If one wanted to, one could mashup (I assume, I’m no coder, but I assume anything can be done) the census block data with a real estate search engine and allow consumers to really target where they wanted to live – and next to what kind of person.

One of the most common search queries bringing people to RealCentralVA is “best neighborhoods in Charlottesville” – what does “best neighborhood in Charlottesville” mean to you? Data might probably will be able to answer this – but humans are prohibited from doing so.

Which brings us to this:

This is a randomly-selected census block in the City of Charlottesville using the US Census’ Population Map (archived: 2013):

Are the similarities between the map above and the map below merely their intended usages?

Related:

– Trialogues podcast discussing big data in real estate with three of the smartest guys in the real estate space (no longer available).

– If you’re really interested, and have a couple hours to learn, spend some time reading Rob’s thoughts on big data (archived: 2021) .

– Your Smartphone Could Be Telling Advertisers How You Shop And Spend