Latest news with #OpenDataPhilly
Technical.ly
11-04-2025
- Business
- Technical.ly
Calling all data nerds: OpenDataPhilly needs your help to get back to its prime
The team behind a longtime Philadelphia data project is calling on users to help improve it — and the city to help fund it. OpenDataPhilly, a catalog of publicly available datasets across Philly, is run by a small team of volunteers who have stayed committed to the site and its mission despite a lack of money and participants. The more people that contribute to its success, the more it can help Philadelphians get engaged with issues they care about across the city, they said. But there's only so much this small team can accomplish as volunteers. With more help and financial support, the site could add features that make the data easier to find. 'There are about [10,000] active users of the site a month,' volunteer Bryan Quigley told 'I don't know what they use the data for, but I hope it makes it a little easier to be civically engaged and make Philadelphia better.' It used to cost $25,000 to $45,000 per year in staff time and $3,000 to $5,000 in AWS hosting costs to maintain the site, volunteer and 'godfather of open data' Robert Cheetham said. Now, it costs a few hundred dollars per year in hosting costs, which Cheetham pays himself, but that doesn't include compensating people's time spent working on it. In the past, charitable orgs like the Knight Foundation have given grants to support the project, but Cheetham found that it's harder to get grants to maintain a project rather than to start something new, he said. 'I wish the city would put in funding to maintain the system,' Cheetham said. 'We've continued to do what we can to keep things running without and do so as we have time as volunteers.' OpenDataPhilly, one of the platforms where the city publishes its troves of data, aims to make that information more transparent, from tracking crime to understanding voter turnout. It was previously maintained by the staff at local GIS software company Azavea. The company was acquired by Element 84 in 2023, which deprioritized the project. Cheetham, who is the founder of Azavea, took the project with him when he was laid off at the end of 2023. Since then, he's been working with a handful of dedicated volunteers to keep it going, including Quigley, a former Azavea employee, who continues to contribute to the project because he cares about the region and the tech, he told Beyond just a data resource, for some of the volunteers OpenDataPhilly provides access to information in a way that builds community. Working on OpenDataPhilly is a way to give back to the city, volunteer Lydia Scarf said. 'We have this sense that there's a lot of people who really care about the project and are getting a lot out of it,' Scarf told 'But I want more and more people to feel like they can be active stakeholders of the project as well.' Losing key functions to stay afloat Before the acquisition, Cheetham and his team made moves to make the site more cost-effective, secure and easier to keep up with. OpenDataPhilly used to be a CKAN site, an open-source data management system, allowing contributors to easily update and share information. However, the system relied on a lot of plugins that were time-consuming to sustain, Cheetham said. The team chose to move to a JKAN site, a lower-cost catalog that stores the links to data. The site is now hosted and managed on Github and using internal tools. These changes made the site more secure and easier to maintain, Cheetham said. 'All of that means the website has no longer got a database behind it,' Cheetham said. 'There's no longer a dynamic programming language. It's simply what's called a static website.' However, the transition came with some feature regressions, like the search function, Scarf said. Now, a lot of the work that Scarf and Quigley do involves coordinating with the city's Office of Innovation and Technology and other orgs to manage pull requests. This means that when the city makes a change to one of the data sets on the site, OpenDataPhilly volunteers need to approve it. Making these changes made the site more realistic to keep up with, especially now that it's a volunteer project, Scarf said. 'We don't have a ton of maintenance to do. Besides that, that's mostly how we're keeping the lights on,' she said. 'But there's a lot to do to get us back to where we were when it was a CKAN site.' A call for more volunteers to get back to ODP's prime The OpenDataPhilly team has a vision for how the site could upgrade its usability, but they need more people to help make it happen. Scarf would like to see a more advanced search function and an updated user experience and design to make it more accessible to people who are using it for the first time, she said. 'The priority in all of this development is around visibility and accessibility, making it much easier to comb through all of these datasets and see what's in there,' she said. 'But [also] to quickly find something that might be relevant to you.' The old site also had a function that tracked how frequently datasets were updated, which Cheetham would like to bring back, he said. He would also like to set up a system that periodically checks to make sure data sets are still available. Ultimately, Cheetham would like to see more data sets from more sources, like academic and nonprofit institutions. A similar mission is in the works at the Economy League of Philadelphia. The non-profit recently got a grant to develop a civic data platform that would make it easier to find and use data. Still, that level of collaboration requires a lot of time and effort. 'There's an affinity of things that could be done,' Cheetham said. 'Without funding and only as volunteers, we're going to chip away at it a little at a time.' Sarah Huffman is a 2022-2024 corps member for Report for America, an initiative of The Groundtruth Project that pairs young journalists with local newsrooms. This position is supported by the Lenfest Institute for Journalism.
Technical.ly
17-02-2025
- Business
- Technical.ly
Philadelphia's open data effort may be losing momentum, but OpenDataPhilly isn't giving up
This article is a part of Every Voice, Every Vote, a collaborative project managed by the Lenfest Institute for Journalism with lead support from the William Penn Foundation, and additional funding from Lenfest, Comcast NBC Universal, the John S. and James L. Knight Foundation, Henry L. Kimelman Family Foundation, Judy and Peter Leone, Arctos Foundation, Wyncote Foundation, 25th Century Foundation, Dolfinger-McMahon Foundation and Philadelphia Health Partnership. This article was created independently of the project's donors. This piece is also available in Spanish, thanks to translation services by Gabriela Rivera, digital communications manager at Resolve Philly. If you want to easily sift through years of Philly campaign finance data or see a map of all the nearest athletic fields, the city's open data visualization tools make it easy. However, a quick dig around OpenDataPhilly, one of the platforms where the city publishes its data, shows some obvious weak spots. Some sources get updated regularly, while others like Police Advisory Commission complaints haven't been touched in years. The city says it's working to make the data updating process more seamless with automated systems and more dashboards to display information. The people behind OpenDataPhilly, a volunteer-run platform that collects open datasets across Philly into one place, say the open data ecosystem faces slowing enthusiasm from the city as administrations change. 'There is essentially still a motivation for the city to maintain open datasets,' Robert Cheetham, 'godfather of open data' and OpenDataPhilly volunteer, told 'But I don't think there's the same kind of energy, commitment [and] resources.' The city established its open data program in 2012 after Mayor Michael Nutter issued an executive order. The initial goal was to publish city datasets in an open data portal, hire a chief data officer, establish an open data working group and appoint a data governance advisory board. The executive order also called for an open government plan. Within a year, the city achieved six out of 10 of its original goals. However, open data has not been as much of a priority for the last few mayors, Cheetham said, and the movement lost some of its energy. By 2019, some of these initiatives, like the OpenDataPhilly platform, remained strong, while others like the working groups and advisory boards had fizzled out. Lack of requirements and automation slow public releases The city hasn't given up on the program, though. It still has a chief data officer, Tim Haynes, and the program still has a system for updating its data with a focus on turning the data into digestible and usable resources, Kistine Carolan, senior program manager with the Office of Innovation and Technology (OIT), told Without a specific requirement for city departments to provide updates to data, though, there's a lot of variance, despite years of progress. Frequency depends on the capacity of each city department, Carolan said. 'The sharing of open data has become more of a habit for city departments as part of their process of having data that they're using internally,' Carolan said. 'That's really exciting because often we have been approaching open data with a more holistic model of sharing the datasets themselves.' Still, the system for updating open data isn't as automated as it could be, she added. To combat this, OIT is in the process of moving its central data warehouse to a new version of data integration service, Data Bridge. This shift will hopefully lead to updated data at a faster pace, she said. Leaving more time for staff to work on projects that present that data in useful ways. 'We're pretty excited about that process both for allowing fresher data publicly, but also for freeing our staff to be able to engage in more projects like these dashboards or applications,' Carolan said. OIT helps make tools like the Campaign Finance Dashboard, which comes from the Board of Ethics and presents visualizations of data from campaign finance reports. The dashboard also links out to the metadata and the raw version of that data on OpenDataPhilly. Another use for the data is 'finder apps,' which help people find resources around the city like primary care providers and recycling centers, Carolan said. The free meal finder app presents an interactive map where people can look up locations for food and meal sites throughout the city, for example. 'It gives an immediate resource to people who just want to explore the data for general trends,' she said. From datasets to dashboards that the public can understand Once OIT has updated data, staff publishes it to various platforms within the hour, Haynes, geographic information officer and chief data officer, told The data is published through Data Bridge to two software platforms where OIT hosts APIs, ArcGIS Online and Carto. Those platforms are posted to the OpenDataPhilly website, the city's metadata catalog, and the Philadelphia website along with its applications, Haynes said From there, the data is available for use by students, journalists, researchers and other residents for projects on anything from voter turnout maps to sharing information about gun violence in the city. OIT then works with city departments to turn their data into interactive dashboards and applications so it's easier to understand. Having open data available is important to transparency and supports city residents working on private data-related projects, but also to improve internal operations and make city services accessible, Carolan said. 'Creating an interactive dashboard where people can explore the data, particularly if there's spatial information related to where they live or where they're working, really allows a broader audience to engage with this data and use it in meaningful ways,' Carolan said. A focus on usability, not just transparency, for the project's next upgrade OIT put out a survey to residents in 2020 to get a better understanding of how people use the open datasets and what challenges they have. In response, it developed an open data dashboard in 2021 to track the metrics around datasets that have been shared publicly. 'You'll see a list of datasets and shows when they were last updated, and if there's a visualization available,' Carolan said. '[It] helps people get a better sense of when something was last updated. That was one of the major pieces of feedback.' The dashboard currently says there have been 552 datasets released, but this includes multiple versions of previously released data. There are also 142 automated datasets and 179 datasets with visualizations. Overall, the city's open data structure has become more 'holistic,' Carolan said. Not only is the data available, but they are working to present it in a useful way that will hopefully improve resident experiences. 'More and more we're being approached by departments themselves who want to build these dashboards,' Carolan said. 'They want to do that both to have transparency about their work, but also they are able to use it more.' Sarah Huffman is a 2022-2024 corps member for Report for America, an initiative of The Groundtruth Project that pairs young journalists with local newsrooms. This position is supported by the Lenfest Institute for Journalism.
