Geospacial Open Data in KY - Louisville v Lexington (Part 2)

Following up on my last post about the state of open data in Kentucky, I want to now do a detailed comparison of every city in Kentucky that has open data online, and give them some sort of ranking.

Right now, only Louisville and Lexington, KY's two largest cities, have any open data as far as I know.  Please leave a comment if you know of another city or have heard another one might be opening their data soon.

There are two types of geospacial open data I'm going to look at, CSV/OMG formatted bulk address-level data, and GIS layer/shape data.  The CSV data is good to pull into a standard database and create javascript based online maps of point visualizations, like we do at Your Mapper.  The GIS data is great for doing offline (though some online) visualizations and creating map-based reports and analysis, like Map Grapher does.

I prefer the bulk CSV data format, because if you want to you can import this into a GIS system, but it's harder to break GIS data in a CSV format for generic database loading.  Having said that, some data is best output in GIS formats, like polygon area outlines (instead of point data).

I'd also like to point out what open data is.  It's not just that it's data on a website in an interactive form, or a PDF to download.  And it doesn't count if it costs money for a complete download.  See the 8 principles of open data for a list, and I'd like to add that historic data is a ninth principle.

Louisville v. Lexington

Right now, the only two cities in KY with an open data portal are Louisville and Lexington.  So it's a UK v. UofL, or Red vs. Blue, matchup for now.  The data are things like crime reports, 311 calls, business licenses, restaurant health ratings, building permits, bike racks, parking meters, property values, recent home sales, vacant properties, and foreclosures, and also GIS layer data.

So let's compare the two in the hopes of fostering some friendly competition and pushing each other to open more data.

Open Data in Kentucky: An Overview (Part 1)

As the national and international movement for open data and government transparency progresses, I'd like to take a moment and discuss the state of things here in Kentucky.

Open Data Definition

First let's define "open data." To me this means that the raw data is available in a bulk download format, like CSV, as opposed to online web databases or PDFs.  These later types of data are more part of government 'transparency' initiatives, rather than open data.  Open data should also be as complete as possible, including historic data, to the extent that privacy laws allow.  

Financial Data

This is the highest priority for government transparency pushes, and with good reason.  There is a great amount of open data regarding government financial records at Kentucky's Transparency Portal, KY Citizen Auditor Initiative,  Louisville's LouieStat,  Louisville Salary and Expenditure data, and other similar data in KY's larger cities.

Public Transit Data

Covington KY was the first to open its public transit feed to the public, with Lexington, then Louisville following.  This allows Google and other companies to absorb public transit schedules and routes to allow the building of direction and travel time applications.  Almost every county in KY has public transit, but most have not made this data available.

Property Data

Property open data is in need of a big overhaul in KY.  Kentucky is unique in the USA because it has county Property Valuation Administrators (PVAs) that asses the taxable values of all property in their county.  Interestingly, these are state-run organizations, with state-level rules and recommendations for opening data and data pricing requests.  Not one of the 120 counties have open property value data. Instead, each charge for bulk downloads and extended property data info.  Some have online text searches, and a few have map visualizations usually run by local or out-of-state vendors, of varying quality and usefulness.

Other Data

Kentucky has an excellent car accident database, and you can grab very detailed open data from it.  Actually, I'd say this is the most complete non-GIS open data set in KY, though the web interface is a bit cumbersome.

The KY Sex Offender database has data available online, but there is no way to download the raw data.  I suspect part of this is because of the State Police's use of a closed third-party tool, which is unfortunate for us all.

GIS Open Data

GIS data is what helps make city government more efficient - it maps everything a city government needs to know about: snow routes, power lines, water lines, property boundaries, street names, and so much more.  But, it's really only useful internally, and so far most KY city govs haven't pushed that data out to visualizations on websites, except Louisville, though Lexington has clunky version.  And no one has opened their GIS data for free download, until Lexington did last week.  Another bright spot is at the state level, where KY has not only visualizations of their GIS data online, but also allows open data downloads.

Geospacial Open Data

Now we get to the kind of data I am interested in and aggregate on Your Mapper.  CSV or OMG Standard formatted raw hyperlocal data that is specific to the street address level.  Things like crime reports, 311 calls, business licenses, restaurant health ratings, building permits, bike racks, parking meters, property values, recent home sales, vacant properties, and foreclosures.  This is where I'd like to draw up some friendly city comparisons in the spirit of cooperative competition.  I'll throw in GIS too since Lexington has done such a good job with it.

In my next blog post, I'll get into a detailed comparison, so stay tuned.

Any other open data I've missed across the state?  Let me know in the comments and I'll update the post.

Lexington KY Publishes Open Data

Lexington KY is the newest city in the world to create an open data portal for its citizens.  Their mission is to promote the openness, transparency and accountability of local government by providing high-value government data in standards compliant, machine readable format. This will serve as the basis for the creation of useful civic applications by third party developers.

From their website:

"For the initial launch, Lexington is providing free access to its geographic information. Over 80 geographic data layers from the city’s enterprise GIS are available and will be updated on a quarterly basis. Links to existing applications promoting the transparency of the legislative process as well as crime data are accessible from the “applications menu”. 

In the coming months we will continue to expand our data catalog and offerings to build on the foundation we have established."

There are 85 datasets in either GIS Shapefile or KML format, great for overlaying on maps on and using offline for analysis.  I talked to Chad Cottle, Director of Enterprise IT Solutions and the chair of Open Data Lexington, via email to get some history.

About a year ago, the local grassroots community (Open Lexington) approached the city asking them to open up their data, etc....the Council passed a resolution placing the city (for the first time) in a position to actually open up data without going the FOIA route.  

So, we brought together our city IT team, the community hacktivisits, the University of Kentucky and other “community movers and shakers” to start collaborating on what kind of stuff that citizens or “geeks” would want.  Working through some user stories we concluded that there were basically 2 audiences.

The first being mom and dad at home who are interested in data viz, infographics, canned answers (like pre-defined queries), etc.  This is also the more app-focused audience.  For example, to use Louistat as a frame of reference, “how much police and fire overtime was there in 2011”?  We will also develop apps to engage the citizen and inform them (think open311, etc.).  On our site we link to two apps/websites that are more “easy access” focused:  Legistar and RAIDS online. 

Audience two are the geeks, hardcore analysts, etc. who want raw, non-manufactured, non-derivative data.  We decided to start with GIS because most of the apps that could be built were location-based or location-aware.  In essence, this becomes the canvas on which the artist will paint. 

The open data portal if for audience #2, the entrepreneurs and data geeks, who can use this raw data to build great things.  The site was built using the open source CKAN toolkit, and implemented by local Lexington entrepreneur Nick Such.   The city paid for it to implemented and hosted in the cloud, so it was very inexpensive, though I imagine it took a lot of time internally to gather all the data and get full buy in, which is always a challenge.

Chad says this is Phase 1.  The next phase starts right  now and will be things like code violations, building inspection data, and 311 calls for service.  Hopefully data visualizations focused around the flow of money in the city will come next too.

Congratulations to Lexington, which now joins Louisville in the open data realm.  My next post will focus on the similarities and differences on both city's sites, services, formats, and open data, in the hope of fostering some friendly competition.  Viva open data!

2nd Annual Louisville Transportation Camp Recap

Last night was the 2nd annual Louisville Transportation Camp, and the event was another huge success!  Over 40 people where there, including Ted Smith, Barry Barker, and representatives from different TARC and city departments, including Metro Technology, were there and provided insights and took feedback.

Also there were local entrepreneurs from Mavizon, Awesome Inc, Insider Louisville, Peak 10, Taco Punk, Forge, Building Layer, MapGrapher, and Roobiq.

After a tour of TARC's Gold LEED Certified building (the only government building to be certified as such in the nation, according to TARC), Ted Smith had some opening remarks, I gave a quick recap from last year's camp and talked about open data and the Your Mapper API, and Barry Barker discussed current initiatives and challenges at TARC, including getting a grant for all electric buses next year, RFID smart payment cards, route optimization, dedicated bus lanes, and budget constraints.

Current Status

Last year TARC opened its GTFS feed to Google and the public, allowing transit directions in all of Google's Map applications, and devs to build tools using the data.   While I'm not aware of Louisville developers that have used the data, there are lots of national services that have picked it up and ran with it, like HopStop, City Maps, and Transit App, and eventually Apple and Bing Maps.  Anyone in Louisville can leverage TARC transit directions within these apps.  City Go Round has another Louisville app list.

Real-Time Data

Finally we moved on to discussing the state of real-time GPS from the busses.  Currently, TARC is using third party software to allow them to manage their bus locations, called Trapeze.  Unfortunately, while this allows them to know the location of all 200+ busses via GPS at the TARC headquarters, the software does not allow export of this live data, for use in the real-time public feed.

TARC is pressuring Trapeze to upgrade their software to allow this.  By December, it is expected to be available on TARC's site on their Trip Planner (powered by Trapeze).  Green Bay Metro already uses this, if you want to see it in action now.

Then by March there should be another update that allows the data to be sent to the GTFS Realtime data feed, allowing real-time bus locations to be known by the public and Google.  This will make bus arrival times more accurate in apps and services, and allow animated maps to be creating showing all the city busses locations at once.

In the meantime, Mavizon offered to place their devices on any number of busses, I offered to make a web interface to visualize their locations on a map, and Peak 10 offered to host the servers, all for at no cost to TARC.  So we'll see if this pilot program idea moves forward at TARC.

Camera Feeds and Open Seats

Every TARC bus has multiple cameras installed, recording events locally.  But transmitting these images to the public is not feasible since they are encrypted by the vendor for security, and are high-res enough to cause bandwidth and cost issues even if they could be transmitted.  So the solution is to install an additional lower res camera and transmit that in isolation.  The goal would be to use software to detect the number of open seats and add this data to the real time apps.

But I believe the cost for this outweighs the benefit for now.  There are likely better solutions for counting passengers, using lasers, pressure plates in seats or exits, and other means.  But these will require some hardware installation and software to transmit the data back in real-time.  And I'm not sure that knowing how full a bus is will affect your decision to ride it very much.  Though it would be neat to see a map of every bus with color-coded markers showing capacity across the city, and a clickable live image of the passengers.

There also could be an app that could accomplish this through crowdsourcing.   A kind of Waze for busses.  Riders could report back on bus conditions, like capacity, noise, smells, mood, issues, etc.  I'm not sure how to get usage of this however, unless it's incorporated into a custom app that provides transit directions (like the ones listed earlier) and other features to keep users engaged.  But an idea worth pursuing for entrepreneurs.

Wi-Fi On-Board

Having Wi-fi access for passengers would be a great perk for bus riders, especially long-ride commuters, who could use the time to occupy themselves on laptops, tablets, iPod touches, and dedicated gaming devices.  One idea was to have some wireless hotspots that could be tested out on some busses to gauge demand.  Another was to partner with Boingo or GoGo to install paid wifi for customers, much like airlines do - that way TARC is not out of pocket and in fact can generate revenue shares from these services.

Summary

Like last year, there was a great energy and enthusiasm on the part of the public, local entrepreneurs, Metro Louisville, and TARC.  The ideas were a bit more focused this year, and seem to be within reach.  The next camp might be as early as March, when the real-time GPS feed is likely to launch.  Thanks to everyone who attended and let's keep the momentum going!

Other Coverage

Insider Louisville
Adam Fish - Forge

Louisville Metro Partnership: Crime Incidents Map

The city of Louisville Metro has partnered with our government open data services and mapping company GovMapper to bring a daily crime incident reports map from the LMPD to the citizens of Louisville.

There are maps for every police division and beat in the city, with data updating daily.  The map customizations also allow the LMPD to include links to specific dynamic maps highlighting recent crimes for their community newsletter alerts.

Launched last week, this is the first time citizens can get daily crime updates from the LMPD with date range and category filtering.  

The service also includes mobile phone access, search exporting, and city-wide heatmaps of crime.  It allows full access to the raw data with latitude and longitude, fostering transparency and open data.  Programmers can also grab the data on-demand for their sites and apps through the Your Mapper API.

Your Mapper also curates a complimentary detailed historic crime reports map going back to 2003, which is updated every month.

Saving the City Money

I credit the office of Mayor Greg Fischer with helping to get this project started (via a tweet in June), and seeing the opportunity for cost savings from public online mapping.  From there Metro Technology Services took over to bring the map to fruition.  They turned to us, the only area Google Qualified Maps Developer.

One of MTS's goals is to save the city, and therefore taxpayers, money across all departments.  They know that by promoting government transparency and open data by putting data online in a way the general public could understand and use would have a huge ROI by reducing the thousands of FOIA requests the city has to fill.  If the public can discover, visualize, and export the data they need online 24/7, then wouldn't have to contact the city for time-consuming data requests.

Another goal of MTS is to create a catalog of raw open data for the public to download and use on their desktop, on websites, and in mobile apps.  To this end they have Louisville's Open Data Initiative, which aggregates data across agencies.  GovMapper has enhanced their raw crime data by adding latitude and longitude to each report, and then providing the entire dataset for download on the public open data site (see CrimeStats.txt).