Category

Data Analysis

Category

In the urgent debate around Seattle’s homelessness crisis, many articles (such as this otherwise great one in Crosscut) cite the statistic that 35% of those who are homeless in the Seattle region have some level of substance abuse. It’s often a very central part of the framing, especially by those who wish to portray substance abuse as a relatively low contributor to the problem. Among other things, presenting that statistic at face-value implies that presumably,…

Last night, an hour-long program aired without commercial interruption in Seattle on the addiction crisis and homelessness. It’s an important watch. I found it devastating, riveting and motivating, all at once. There is already much being made over the fact that (a) it comes from KOMO News, a station now owned by Sinclair Broadcasting, a large conglomerate which has an unquestionably conservative lens. And that (b) there is direct footage within this broadcast of people…

The Washington Post’s Jeff Stein estimates that raising the marginal tax rate to between 60 and 70 percent on incomes above $10 million might raise as much as $720 billion dollars over a decade, or $72 billion per year. There are some 16,000 households that meet that criteria — fewer than 0.05% of all US households. Collectively, their taxable income was about $405 billion in 2016, on which they paid $121 billion in taxes. Current…

In WWII, researcher Abraham Wald was assigned the task of figuring out where to place more reinforcing armor on bombers. Since every extra pound meant reduced range and agility, optimizing these decisions was crucial. So he and his team looked at a ton of data from returning bombers, noting the bullet hole placement. They came up with numerous diagrams that looked like this: Most of his team members observed “Wow! Look at all those bullet…

I’ve just discovered an incredibly cool project on Github: DeOldify, which uses deep learning to automatically colorize old black & white photos. It’s not perfect, but what it’s able to do is pretty amazing, and improving rapidly. In addition to ninja-level coding, author Jason Antic (@citnaj on Twitter) does a terrific job writing up how the algorithm works in the README file. Essentially, his code uses a deep learning technique called a Generative Adversarial Network (GAN.)…

Jupyter Notebooks are a great tool for data scientists, allowing you to work with languages like Python and R to analyze and visualize data. I was pleased to see that The Azure team has now made Jupyter Notebooks available for free at https://notebooks.azure.com. Here, in a simple “Hello World” application, I’ve used the platform to do a simple analysis of the bicycle traffic over the Fremont Bridge here in Seattle. There are sensors on the…

I was there at that raucous council meeting to repeal the vote, I signed an Open Letter from Tech Leaders Opposed to the Proposed Head Tax, and I was a volunteer signature gatherer and small donor ($275) to the repeal referendum effort. Why did the council vote 7–2 to repeal it, just a few weeks after voting 9–0 to pass it? It is telling that the City Council called for a vote to repeal…

King County’s Restaurant Inspection Grades, which is based upon the average of red-card violations in the past four inspections I’m exploring the Python stack for data analysis and machine learning. I know I’m late to the party, but have only recently discovered the impressive Jupyter Notebook (formerly IPython) data analysis platform and community. It makes “storytelling from data” easy. But doing so with ease requires fluency with a sometimes unintuitive yet very powerful syntax. Today’s…

I’ve been dusting off my machine learning/data-science skills by diving into Python, which has become a lingua franca (along with R) of the data analysis world. Python’s libraries for data analysis and visualization are really superb and can make quick work of complex data analysis tasks. Sentiment Analysis Today, it’s possible to use computers to quantify, with reasonable accuracy, the emotional “sentiment” of an utterance, determining if it is fundamentally positive, negative or neutral in…