I'm not a writer, I'm a coder (not great at that either).
But what I do know is that there are reams of data hiding just out of reach of the average internet user. And that's where my coding skills come in, I've been scraping data out of South Carolina government operated systems since the beginning of the COVID-19 pandemic, and I fell into a huge palmetto shaped data mine.
I've built a couple of twitter & mastodon bots. One of them posts orders and opinions from the South Carolina Supreme Court as soon as they are released. I think some journalists find the bot useful! I also wrote code that tracks (takes a little while to load) South Carolina Department of Health and Environmental Control Health Inspection grades. It posts the worst ones to Twitter.
Most of the datasets I'm interested in are of public interest but cannot be easily understood in the systems that they reside in. For example, one of the data sources I've been researching and exploring powers the South Carolina Ethics Commission's campaign finance reporting system.The website is great but the user is not going to gain much insight at all from looking at individual contributions or campaign disclosures, but I've been able to write code to dump the data that the system holds and that is much more useful.
In this blog, I'll share my experiences and insights on how I navigate the world of hidden data through coding. I'll discuss the challenges, techniques, and tools I use to extract and analyze data that's not easily accessible within existing systems. From exploring datasets from the South Carolina Ethics Commission's campaign finance reporting system to random bits of knowledge that I uncover during my exploration. While I'll primarily focus on South Carolina-related topics, I might also delve into unrelated topics in some posts. I'll try to keep you from falling into the mine like I did.
This blog is free and will remain so, as I have no interest in monetizing my work.