Friday, November 15, 2013

Press Roll Change



As I was doing a tour of the Statesman for some St. Edwards media communications students today, I remembered that I created this video a little while back. It's worth sharing. It's cool technology, but old school.

Saturday, November 09, 2013

Texas A&M Geocoding services saves the day

In my inbox yesterday landed a list of 650 damaged houses from the October 31st flooding here in Central Texas. It had street address, but no zip or city. I got excited when I saw PLACE_ID, because I thought I could use that to match a parcel shapefile that I received from the county tax assessor's office.

Well, never assume. When I got to merging the files later in the day, I found that PLACE_ID is not the same as PROP_ID, and I was screwed.

I then remembered that Texas A&M GeoServices has a batch geocoding service. I'd tried it before with a smaller list, but didn't have much success for some reason I couldn't remember. So I tried again.

My original file had just a "Full Street Address" like 10000 WILD DUNES DR. I knew some were in Austin and some were not, but I just added columns for City and State and made them all Austin and ran it through the batch geocoder to see what would happen.

It took about 3 minutes process the 650 records, and it spit back all kinds of good information beyond the Lat and Long, mainly the "GeocodeQualityType." I knew from my geocoding experience with the Bastrop fires several years ago that "CityCentroid" wasn't a good result, and that was true for about a third of them. I was really looking for "ExactParcelCentroidPoint."

Since many addresses were on the similar streets, I manually added a ZIP to all the addresses and ran it through again, with *much* better results. All but one address was either "ExactParcelCentroidPoint" or "AddressRangeInterpolation" which turned out to be pretty good.

The Google Fusion Tables map is below, and this link is to the entire interactive.

Thursday, October 24, 2013

Beer selection on an X and Y axis


So, this isn't much to do about anything except my love of beer and data visualization. Above is the beer menu at Wright Brothers Brew & Brew, a coffee and beer bar in Austin.

The X axis shows how hoppy a beer is, while the Y axis shows how dark the beer is. Very helpful in ordering.

Saturday, August 17, 2013

Using Google Charts in Caspio

When the Texas Education Agency released school accountability ratings late last week, we turned around in the same day a searchable database to compare results for Central Texas. We had started with a PDF report of pass/fail figures, so it was a bit of a challenge to get launched, but not anything new or noteworthy technically (though useful for parents of school children in Texas.)

Right as I was leaving that day, we found the data download for the full results that included individual scores on four specific indexes the TEA based their ratings on. I went back into the project this week and updated it to visualize those scores using Google Charts javascript library, which was a new effort. I'd used Google Charts as an image, but that's being deprecated.

There was one challenge worth sharing: My data often had blank values for schools that weren't rated for that value. When those blank values were returned into the chart (I choose the gauge to show the values), it would break all the javascript for all the charts. I didn't want to use zeros because I didn't want to show failing scores for schools that weren't really measured. So, I had to evaluate the value each time I used it as a variable, and then choose whether to build or ignore the gauge accordingly. This is not rocket science for a programmer, but it was something I hadn't done before.

In my edited code snippet below, which would show one gauge:
  • CI1 is variable I end up using from the Caspio record to build the chart.
  • CI1Value is a variable I use just to evaluate if CI1 is blank. If it is, I make CI1 an empty string so the rest of the page doesn't break. If it has a value, I change it to number and assign it to CI1.
  • CI1Cut and CI1CutValue are similar, but this is another value I pull from Caspio. It's the "passing" rate for that index.
  • The [@field:] code is where I'm calling the data from Caspio.
  • Lastly, at the bottom, I evaluate whether either CI1 or CI1Cut is a blank string, and if it is I skip writing out the div. Otherwise, it gets drawn and all is peachy.
I'm sure there is a better way to do this ... like I could probably write this without the CI1Value variable and just do those evaluations with CI1, but it worked, and that was important at the time.

There is a lot more to do with this data. I'd like to write the search results better so readers can more easily compare schools, and there are also lot of the tables and values that TEA uses to come up with the indexes that would be great detail for each school. I'll have to come back to it in the future.


Tuesday, July 02, 2013

Getting more U.S. Census cities into Tableau

I've been working on a visualization about commuting populations and came across an issue getting cities to map in Tableau. My data included 381 "places" (cities and other similar municipal entities) in Texas, and the "City" geographic role that is baked into Tableau would only match with certainty about 150 of them by their name alone.

In the past, I've used Census "places" shapefiles in Google Fusion Table maps, and that would have served me well for a map alone, but I really wanted to pair this map with some bar charts that would work together.

So I had to find my own latitude and longitude for my Texas places, and while this worked, it took sufficient enough effort that it's worth explaining here. This isn't the same a having geographic boundaries for each city, but since I wanted to do a bubble map, this wasn't a problem.

The first challenge was to find a list of cities and their lat/longs, preferrably from a U.S. Census source so I could match with my data. The Census delivered with the 2010 Census Gazetteer Files. I found I could get every Census "place" in Texas, along with the 2010 population and lat/long for each one, all wrapped in a cute tab-delimited bow. Booya!

Now, I just had to merge that lat/long from my Gazetteer file into my commuting data file. Might seem easy enough, but there were 1752 records in the Gazetteer file compared to my 381 in my commuting data. A manual sort and copy/paste wasn't going to do it.

Microsoft Access to the rescue. I used a common field in both tables and join them, and then created a query with the fields I needed from both tables. This is the subtle power of Access that you can't get with Excel.

First, let take a look the important parts of my original commuter data:


I know from working with Census data that the "State" field -- the two two-digit value 48 -- and the "Place" field is part of a FIPS code designation used in almost all Census data. This is a good thing.

Now let's look at the important parts of the Gazetteer data:


The "GEOID" is indeed analogous to the "State" and "Place" fields in my commuter data. (I also have here the "INTPTLAT" and "INTPTLONG" fields, which are latitude and longitude values.) I used a simple string formula in a new column to combine my "State" and "Place" fields:

=B2&C2

That concatenates the two fields. "48" and "01000" become "4801000" and match my GEOID from the Gazeteer data. I also called it GEOID.

I did some further cleanup to my original file, getting rid of "city", "town" and similar suffixes from the "Place Name" field. I imported the two tables into Access, naming them CommuteList and PlaceList.

I write my Access queries in the SQL View because that's how I learned. (Someone needs to teach me the Design View method.) A simplified version is this:

SELECT CommuteList.PlaceName, CommuteList.PopEst, CommuteList.WorkEst, PlaceList.Latitude, PlaceList.Longitude
FROM CommuteList, PlaceList
WHERE CommuteList.GEOID = PlaceList.GEOID;

Looking at the WHERE line first, I'm joining the two tables by their GEOID fields. Using this methods, I'm only going to get results where the same GEOID exists in both tables. The SELECT line pulls the specific fields I need from each table.


Now each place has it's own latitude and longitude, so I can export this query as a new Excel spreadsheet and then use this to build my Tableau map and I'm assured all my cities will be included.

I'm still working on the visualization and it will publish later this week, and I'll update this entry when it does.

Post updated with link: Interactive: Commuter-adjusted populations in Texas.

Thursday, June 06, 2013

Travis County gun deaths and using visualizations to shape reporting

In the wake of the Sandy Hook shooting in Connecticut and the resulting heightened sensitivity to gun control, Statesman reporter Eric Dexheimer requested the medical examiner's investigative report for every gun death from 2010-2012.

In the resulting analysis, Dexheimer found that two thirds of the gun deaths were suicides, and that many of those suicide victims had known mental illness diagnosis.

Shortly before the story ran, I asked Dexheimer for the data he had collected, hoping I could piece together a visualization to accompany his story. After deciding to concentrate on the age, type of gun and type of death, I was able to do some interesting things using Tableau.

Probably the most important thing I learned was I should've done this earlier. Dexheimer and I had talked that the geography of the deaths would probably not be important, but once mapped it was clear  that the majority of homicides happened east of I-35, areas that have higher poverty and crime rates, and the majority of suicides were west of I-35, considered more affluent  That might not be surprising, but it was certainly telling, and worth further reporting. Unfortunately, we were already upon the deadline.

While Tableau may not be intuitive, once you get the hang of it you can do a lot pretty quickly ... certainly quick enough to do explore your data for trends and outliers while you are reporting.


Monday, May 13, 2013

Election results by precinct, AISD bonds

When I heard we were doing a precinct-level map of election results for print, I figured I really needed to do one online, where readers could really study it. I had the county clerk send me an example file so I could play with it and figure out how I might show the results, and prepared to do the map on Mother's Day, the day following the election. Print had their analysis story pegged for Monday's edition.

When I saw how close the votes were on all the proposals (they each passed or fell by fewer than 1000 votes), I wanted to highlight how close those races were, and where they were closest. I choose to highlight just the closest 5% to center, which would've tipped the scale either way.


Tableau: Police calls at Rundberg motels



Finally got to use some of my newly-minted Tableau skills on this visualization on police calls to Rundberg-area motels. It's a bit of a challenge since we are tracking almost a dozen hotels, so things like sparklines don't show that well until you change the views you are looking at. Jewel xxx at Tableau helped me with the map concept, for which I'm grateful.


Special thanks to Jack Darby of krimelabb.com for supplying the data.

Sunday, April 21, 2013

Thoughts from ISOJ 2013

First, let me say that Rosental and Amy put on a fabulous conference, yet again. Great topics, fantastic speakers, and everything else worked.

I'm not going to do a deep dive on this ... I just want to throw out a few thoughts on a few of the subjects.

Chris Gilbert was brilliant in his discussion about how the Deseret News followed the Clay Christensen model of disruptive innovation to not only survive, but thrive. They split off their digital business from the legacy print business, to give it independence and flexibility to do what was necessary to grow. (Big over simplification, but heh.)

Following that was more panels on disruption, with David Skok of globalnews.ca, Jennifer Carroll of Gannett digital, Jim Brady of Digital First and Jim Maroney of Belo and the Dallas Morning News.

For me, I related most to Jim Brady with the Thunderdome project (name alone wins my heart!), which is essentially shared services for digital, but it sounds like they did it right. Would really like to hear the tale from a foot soldier inside that org. Maroney was almost a polar opposite of Gilbert from the previous panel, touting new business ventures to prop up the not-flattening losses for the print and digital mediums. {sigh}

Andy Carvin's talk on social media was definitely on point, as that morning he had been working his social media mojo to find what was "most probably" Dzhokar Tsarnaev's Twitter account. There is a lot of talk about incorrect tweets, scanner traffic, reddit, and the word "confirmed." At points I wanted to shout "Preach it!".

This was my second time to hear Trei Brundrett of Vox Media (SB Nation) talk about their development efforts, this time focusing on responsive design. I was (and am) enamored with their "living story" idea, and tried to talk my company into something like that a couple of years ago, but we didn't have time to devote that development effort. Of course, Vox Media redesigned and recoded 310 sites in 6 months, but hey ... whatever. Miranda Mulligan talked about her experience at the Boston Globe, who introduced one of the first major responsive news sites. Michael Donohoe talked about Quartz, and the Texas Tribune's Travis Swicegood talked about his thoughts how responsive design should be about more than changing width and device, and more about the person viewing the content.

Lots and lots of fun. I really like listening to these technical panels and the challenges these guys have faced and conquered. Makes me feel a little small that I'll never be in that league, but I guess I have a different value in this world. I hope.

Day two started off with Emily Bell, formerly of The Guardian. I was a little more interested in the next talk on mobile journalism, where Ivo Burum and Alisa Richardson both talked about their incredibly inspiring work to get mobile journalism into the hands of others who don't normally have the opportunity. Representatives from three heavyweights, Chicago Tribune (Chris Courtney), The Wall Street Journal (David Ho) and the Washington Post (Joey Marburger) discussed their mobile strategy and development efforts. Courtney's talk hit home as he talked about development mistakes that I've helped my company commit in past and present. All three of these guys are brilliant.

The data visualization talk was both inspiring and disheartening, as there was some really good work shown and I would like to live up to that. Alberto Cairo's talk on the quality of graphic presentation really had me thinking that I need to make sure I have a little more theory of good design in my next dataviz class. Following was one of the graphic editors in Snow Fall, an excellent piece of "immersive storytelling" that earned a Pulitzer. More inspiring work by Propublica's Scott Klein and Periscope's Kim Rees.

Jill Abramson of the NYTimes closed it out for me (I didn't stay for the research papers) and it was cool to hear she was in the newsroom most of Friday night when the 2nd Boston bombing suspect was caught. She does run an incredible staff (four Pulitzer's don't lie) and she is sure to give the credit for it.

My best discussions of the weekend, as usual, came outside of the actual conference. I got a lot of great higher education insight from Mindy McAdams (University of Florida, Patrick Howe (Cal Polytechnic), Jonathan Groves (Drury) and of course, Cindy Royal of Texas State.

There was so much going on, and I didn't capture any of it very well. Made me feel good to be journalist in this day and age, and I hope to live up to the standards folks on the panels and in the audience set every day. Awesome.

CLARIFICATION: I had described Chris Courtney, David Ho and Joey Marburger all as developers, and that was an oversimplification and really just wrong. Their titles listed are:  Chris Courtney, mobile product manager, Tribune Company; David Ho, editor of mobile, tablets and emerging technology, The Wall Street Journal; Joey Marburger, mobile design director, Washington Post.



Saturday, April 06, 2013

Using calculations in SQL statements

I've not done much training with Microsoft Access, so I've been learning by doing and trying, for the most part. This week, I figured out that you can do a calculation in SELECT statement, like to figure out the percentage of records in your selection. Up to now, I've pulled grouped results into Excel and did my calculations there.

I would think there would be an even easier way to build this, but ... maybe there is if I knew how to use the Query Wizard instead of writing my own SQL.


SELECT AASStatus, COUNT(*) AS [COUNT], FORMAT((COUNT(AASStatus) / (SELECT COUNT(*) FROM [DC-Match-N])),"Percent") AS [Percent]
FROM [DC-Match-N]
GROUP BY AASStatus;

So, my table name (actually a query) is [DC-Match-N], and I want to group the all the records by their AASStatus, count the records and get the percentage. Probably my most common goal is to find out how many records meet a certain search criteria.

Breaking down the SELECT LINE:

  • First is the field I want to count.
  • Next is the count of records for that row. Since I GROUP BY AASStatus later, it will count the records that match each status. I aliased this column as [Count]
  • Next is where we do the percentage:
    • FORMAT allows me to format a result as a percentage, currency, etc. I learned about it in this post on technonthenet.com. (I actually did this first by doing my own *100 and using the ROUND function, but found the FORMAT function later.
    • Inside of the FORMAT tag is the math. Put simply is it "[part] / [whole]" formatted as a percent.
      • The [part] is the COUNT(AASStatus), or the count of the records that matches the AASStatus for that row.
      • The [whole] is a new SELECT statement to get the count of all records from the [DC-Match-N] query.
      • The percentage part is handled by the FORMAT function above.

Here's what it looks like:


DC-Match-N-AASStatus
AASStatus Count Percent
Status 1 136 4.63%
Status 2 39 1.33%
Status 3 5 0.17%
Status 4 190 6.46%
Status 5 2569 87.41%

This seems so simple ... to do the percentage as part of the select, but I hadn't thought of trying it before. 

Sunday, March 31, 2013

Love the Google Treasure map!

Today, Google Maps has an awesome Easter Egg (on Easter) or April Fools joke ... the Treasure Map view:


I particularly like how there are bats flying out of Austin:


Hat tip to @fmashood. (Post updated to note April Fool's link instead of Easter. I still think it's a good Easter Egg.)

Automatic legends on Google Fusion Table maps

I'm not sure if it was added recently or if it has always been there, but one if my students stumbled across an automatic map legend for Google Fusion Tables. It's been a long-time frustration that I couldn't add a legend to explain the color variations in my maps, one quickly discovered and lamented by my students.

You can find the Automatic legend under Change Map Styles
Then the other day as I was grading projects for my data visualization journalism class, there was -- VIOLA -- a legend on a FT map! I exclaimed joy. I jumped up and down. I BEGGED to know how she did it!

It was an accident, but a damn fine one.

So where is it? When you are in Change Map Styles, under where you change the styles for your points and polygons. It's labeled "Legend," and there are options to set the title and where it displays on the map.

You can see an example of it's use in my Statesman map showing geographic mobility in the Austin MSA.

I wish there was more flexibility, of course. I'd like to be able to add a "%" sign after my values in the above map, for instance. I guess I could reconfigure the data and make the column a percentage. I'll have to think about that going forward.

The legend will pull in your polygon fill colors and values.

Wednesday, March 13, 2013

SXSW Tuesday: I'm tired. Of. It.

OK, so motivation is a little harder to come by today. Cory and I had a great time at the SXTxState Taco Party and the Statesman Social Media Awards last night. And now to get through the last day before it's back to work for a couple of days.

Pandora and Techcruch

A talk with Tom Conrad, one of the leaders of Pandora music service. It was interesting, but I won't spend a lot of time here on it (or on anything today).
  • Pandora thinks their future still lies with advertising models because of their scale of listener base, unlike Spotify, which is subscription based more than advertising.
  • Even though their CEO is leaving, the company is poised for greatness as advertising dollars are starting to shift to mobile.
  • They didn't like putting on the 40-hr cap on web streaming when they did, and removed it once their monetization model caught up. Same goes for the mobile streaming limit, which is fairly new. After 40 hours in a month, they will charge a $1 for the rest of the month.
  • As they've faced challenge after challenge, and competitor after competitor, their focus has been to turn back into the product and make it better.
  • In a discussion of House of Cards style commissioning of content, Conrad joked that they should get the cast of Glee to record every song in their catalog and they would offer that.

Getting ready for the job that doesn't exist

This was Cindy Royal of Texas State Journalism and Aron Pilhofer of the New York Times talking about the kind of people and skills needed for future jobs that don't yet exist.
  • College is good for teaching you how to learn. Don't depend solely on it.
  • Have a passion and pursue it and do it well.
  • Pilhofer says not enough people come out of college with a specialty they do really, really well. Too many generalists. Have a passion for something, and it will show.
  • Never underestimate the value of being able to write a sentence. It helps with any profession.

Scrum as theater

This talk by Gwydion Suilebhan was actually very interesting, talking about the similarities between the Scrum agile development methodology and Devised Theater. Being familiar with Scrum from my previous work as the Technical Solutions Manager at the Statesman (a role I loved and hated to see scuttled) I thought this would be a good way for me to go get reinspired about agile thinking and find a way to bring it to my daily work.

Devised theater, which has been around for 40 years, bucks the trend of traditional (read waterfall) development method of theater where a playwright writes at script, then gives to an Artistic Director who gives to a Director who gives to designers and then actors to perform. This puts the playwright further away from the finished product.

In devised theater, the script is something that comes together as a collaborative process with the director, actors and designers. In fact, it may not get written at all until the work is "done" and someone else might want to perform it. The audience is brought in very early to see scenes and previews performed and their feedback is sought and watched and incorporated into the play.

A couple of thoughts to come out:
  • Suilebhan suggests the first minute of a Scrum standup be used for warmup exercises. It gets everyone engaged and ready to move on.
  • A talking stick could be used to help control some of the team dynamics of shy and talkative people.
  • Almost every tenant of Scrum -- user stories, themes, epics, review meetings and retrospectives, backlog grooming, individuals and iterations over process and tools -- all have equivalents in devised theater.

Bruce Sterling

Another case where trying to synthesize what someone says is a challenge. Sterling is always a pessimistic treat to end the festival, foretelling the doom of our age in general and Austin specifically through different metaphors.

This year he compared the South by Southwest tech scene of 2012 to the ancient Sinagua Indians tribe of the Southwest, the tech leaders of their time. Here's hoping we don't end up the same way, with building shells and a legacy of what ifs and what the Fs.



Monday, March 11, 2013

SXSW Monday: Kids ... drama and games

I had to catch up with some stuff at work, so no 9:30 a.m. panel for me. My first was at 11 with a talk on responsive design. Dave Rupert cracks me up.

WYSIWYS: What you see is what you spec

A talk on responsive design by developers from The Times (of London), Paravel and Code & Theory. The panel mainly talked about their approach to the challenge and they all touched on creating modules that you could reuse.

Dan Gardner of Code & Theory broke their approach to responsive design this way:
  • Purpose
    • Will the content focus more on text, commerce or image. Having that perspective can help in what are the most important parts to tackle in responsive design.
  • Platform
    • Will you be native or not? Same design with a different wrapper per device?
    • consider the technology (cms) drives content and fucntionality. Will you have three headlines depending on devices.headlines for devices). Is there functionality you can add to add value, gestures and such.
  • Prioritization
    • It's not mobile first; it's all platforms at once.
    • Tackle from the ends and work toward the middle ... the big layouts and the small modules
    • Prioritize your mid-points and breakpoints. Consider time to market, longevity of experience (a sxsw app for only a week?), resources, target device, complexity of behavior.
  • Process
    • Code & Theory is constantly refining their design and development process, even after many RD sites.
    • Argue early and often. Prototype early and often.
    • They use Indesign and Keynote for most of their UX/prototyping
    • Design is not done until it is developed. And even then it isn't done.
Alex Breuer, The Guardian actually talked more about his work at The Times, where they created a special responsive design. Their tablet experience is a responsive design experience that they were able to adapt to a new kindle in hours instead of weeks. They created a very complicated javascript layout engine to decide how to display the content. It makes decisions based on the number and quality of photos, fills whitespace with fillers like pull quotes and such. It is the strength of their app/RD experience.

And then there is Dave Rupert. What a loon ... I love him. He really focused on the modules approach. In a site design like Mashable, they took those pieces that they spent so much work on for the home page and resused those modules at the article level so that great design isn't wasted, since many users get directly to the article level from social sharing.

A couple of mentions:
  • http://smacss.com/
  • http://twitter.github.com/bootstrap/
He talked about websites being a brand identity system much like a logo-to-lifestyle like Westinghouse. He talked about building a pyramid, with the base on the bottom. (So read this bottom to top.)
  • Theme
  • State (:hover)
  • Module (.module)
  • Layouts (grid and structure)
  • Base (reset & type default)


That's where bootstrap came in. It's a way to get there (though not always THE way.)

Digital Drama: Growing up in the age of Facebook

This panel was about self-preservation. I have two high school-age boys who live in this world and I can use all the help I can get. Though, to be honest, the panel was mostly dominated by a fast-talking researcher and about-to-be author (Danah Boyd) who spoke just a smidge above my head (and probably others) and a Slate reporter (Emily Bazelon) who has written extensively about Phoebe Prince, a teen who committed suicide. Then there was Bill Keller (NYT ... has a 16-year-old daughter) and Jason Rzepka of MTV, which has an app that helps teens figure out the thin line between innocent and inappropriate.

Here's the snippet:
  • What most adults think of bullying, kids think of as drama.
  • Boyd thinks media has blown up the cyberbullying thing so much that caused lots of laws and rules that educators and parents focus on instead of the real issues of help young people negotiate their relationships.
  • Keller says society has seen a general increase in meanness, aggress and polarization, but he doesn't blame the Internet, because Rush Limbaugh doesn't need the Internet to be a son-of-a-bitch.

Keynote: Julie Uhrman of OUYA game console

Uhrman has a lot of energy and I think the open developer system will be really interesting for the TV/Set-top box game system, but I'm going to leave discussion of that to a much better source in Omar Gallaga, because he is awesome and a gamer. I'm not.

Up tonight: Statesman Social Media Awards

Heading tonight to the TxState Taco party and the Statesman Social Media Awards.



Sunday, March 10, 2013

SXSW Sunday: Love what you do; do what you love

Having gone without a real day off since Feb. 17th, I'm kind of burned out and it's hard to keep focused on the festival. It didn't help that arriving exactly at start time for my 9:30 a.m. panel, there was a line outside the venue much larger than the venue could hold. That was my 11 a.m. venue, too, so I bailed and went to see Amit Singhal talk about Google Search.

Amit Singhal and Google Search in Mobile World

Amit was clever enough and a good chat, but he didn't reveal much insight into Google Search. Clearly, he has a passion for search as a way to change the world, and for it to be perfect for every person.

The prefect engine should know exactly what you mean and give you exactly what you want. - Amit Singhal.
Singhal wouldn't reveal much about the search engine algorithm or any secrets in "gaming" it (of course.)

  • Inbound links are important
  • So are a good title and the content
  • There are over 200 search signals included in the code
  • The code base is very large. More than can be written in a day.
  • They don't manual change rankings for much. Porn. Legal reasons. They will change the algorithm instead.
  • Google+ allows Google to include personal information in search returns ... information that isn't available without those connections.
Clearly what drives Singhal is doing good in the world. He told a story about a farmer in Africa who had an ant problem with his potatoes. He was able to use an internet kiosk to discover that he could spread ashes on this crop to fend off the pests, and had a bumper crop.

Google cares about non-developed countries, translating web pages to offer the entire web to those countries instead of the sliver they would have available in their native language.

Google Now looks at answering questions before you know them. That your flight is delayed. That you should leave earlier for your next appointment because there is heavy traffic.

Singhal's career advice: Follow your heart and do what it says. Because if you do, you will sleep happy, and happiness is worth much more than any amount of money you can make. 

Tina Roth Eisenberg keynote

Eisenberg talked about the 11 rules she lives by, with examples of how they have related to her many passions: her design blog, her design studio, Creative Mornings lecture series and Tattly (and probably more.) 
  1. Invest your life in what you love
  2. Embrace enthusiasm
  3. Don't complain; make things better
  4. Trust and empower people
  5. Experience is greater than money
  6. Surround yourself with like-minded people
  7. Step away from ego and collaborate
  8. Ignore haters
  9. Make time to think and breathe
  10. If an opportunity scares you, you need to take it
  11. Be an eccentric aunt to someone else
Eisenberg was fun and I like her energy and passion. Again, Love what you do. Follow your heart.

Nate Silver

Basically Nate doesn't understand why everyone is so exited about what he did in predicting the Presidential election, because what he did is pretty easy:

The 538 method:
  • Average the polls
  • Count to 270
  • Account for the margin of error
It's that last point that mattered. Other folks got pretty close ... 49 out of 50 states. And even in Silver's case, Florida was a coin flip anyway.

A couple of other interesting points:
  • 90 percent of the data in the world was produced in the last 2 years.
  • This is not the first time we've had that. The same thing happened after the invention of the printing press in the 1400s.
  • The world is more polarized now, and that trend seems to coincide the mass media.
  • Again, that happened with the printing press, too. There was more conflict in the 100 years after the printing press than any other time in history.
  • Silver brought up the 80/20 rule like Tim Ferriss yesterday. That 20% of effort will get you 80% accuracy. But it is the profit margin that is in the other 20% of accuracy. Take that, Tim!
  • Silver got into stats so he would win his fantasy leagues.
OK, so Nate didn't tell anyone to follow their passion, but clearly he does.

Craig Newmark and Kelly McBride

OK, here's the deal. I'm here at this panel now, but I'm not listening. I'm instead installing LAMP on a new server I got for free from DigitalOcean, hoping to use it to set up DocuWiki. Wish me luck.

Trade show

Which leads me to the trade show. I spent time at Tableau and Digital Ocean. Ate AP's breakfast tacos. Tried the symbol-less dasKeyboard and scored 73 words a minute.

What I really wanted was to find Google Blogger to answer a question about one of their templates, but when I searched for them on the convention index, they did not have a booth. There were no Google booths.

I guess they don't need to market.

    Saturday, March 09, 2013

    SXSW Report, Saturday: Perfect Coffee

    TV and Twitter

    Started the day mistyping Jenn Deering Davis's name and Twitter handle about three different ways during her talk on "How Twitter has changed how we watch TV", but I did enjoy her talk to a packed house.

    The takeaway: Could Twitter traffic be helping networks get more people to watch shows live to avoid spoilers?

    Meta-learning

    Tim Ferriss talked about the 4-hour ethos: accelerated learning for an accelerated time, and his book The 4-hour Cook (a huge tome). Not being familiar with his past works (4-hour work week, etc.), this was all new to me, but in essence he threw out this: DiSSS. Break down your skill and focus and you can become an expert in 6 months.


    • DECONSTRUCTION: Break the skills into small, doable pieces. Swimming is complex, but break it down to kicking, arms, breathing.
    • SELECTION: Find the most valuable stuff to learn. 4 cords in almost all pop songs. 13 sentences can help you learn any language.
    • SEQUENCING: Change the sequence. To learn dancing, learn your partner's role first. Learn closing moves of chess, not the opening.
    • STAKES: You need incentives to do well. Make them mean something.

    Other tidbits:

    • Focus on subtraction and not addition. "Perfection is achieved not when there is no more to add, but when there is not more to remove."
    • The perfect cup of coffee: Use 12 grams of fresh hand-ground coffee (from burr conical grinder) with 120grams of hot water not to exceed 180 degrees in an Aeropress.
    • Ferriss practiced his 2007 SXSW presentation in front of his friend's three Chihuahuas until he could sustain energy long enough for them not to walk away.

    Elon Musk

    This is rocket science. That's the first thing I though when I walked into the packed simulcast room (yes, event he simulcast was packed) for this leader of SpaceX, Tesla and SolarCity.

    My main takeaway is this man is smart and busy and probably neglects his five children. OK, not really, but he does live in a different world than me, and frankly he should.

    SpaceX wants to build a spaceport in Texas. The Open Beaches law hinders that because they want to be on the coast but you can't keep people off the beach. He thinks it will happen, though.

    Musk was pretty unrepentant about his reaction to the NYTimes review of the Tesla S model. He said the only think he would do different is now he should publish the rebuttal to the Times' rebuttal. He says he has no problem with critical reviews ... he has a problem with false reviews.

    He offered to help Boeing with lithium batteries because Tesla knows them very well. He offered the help because his friend, Richard Branson (Virgin Airlines) was losing millions of dollars with the rounding of the 787.

    Ten positions of basketball

    This was a really interesting panel by Muthu Alagappan with help from Jeff Becham about Alagappan's work using topological data analysis with basketball statistics. Topological visualizations map connections between multiple dimensions (stats in this case) in a multidimensional model that shows connections. With this, Alagappan has found there are really 10 common positions in basketball, instead of the traditional five positions.

    Traditional positions:
    • Point guard
    • Shooting guard
    • Small forward
    • Power forward
    • Center
    Alagappan's positions
    • Inside-Outside Scorer
    • Mid-range big men
    • Two-way all star
    • Jump-shooting ball handler
    • Defensive ball handlers
    • 3-point specialists
    • Low-usage ball handlers
    • 3-point ball handlers
    • Paint Protectors
    • Scoring Rebounders
    The example he used were "point guards" Chris Paul, Steve Nash and Jason Kidd. They are all considered point guards, but they all have completely different styles of play. Paul is a mid-range shooter, while Nash is a consumate pick and roller. Kidd can post-up but also shoots the 3 like crazy.

    Thinking of players in this way frees up teams to expand their thinking about players and rosters. Alagappan has worked with NBA teams and owners, and says the stats teams collect vary widely.

    Another part of their research uses the newer SportView cameras that catch and track players at every angle of the court.

    I closed out the day at the Awesomest.Journalism.Party.Ever were I caught up and spotted colleagues and friends, like Jen Lee Reeves, Amanda Zamora, Rosental Alves and more. I touched base with Tableau and checked out Atavist, a publishing platform for long-form non-fiction.

    Still, the best conversation of the day was a quick, unexpected one in the Roku lounge with Reuben Stern, a professor at the University of Missouri School of Journalism. Since he works on multimedia, I talked with him about my video explainer thoughts for my data visualization class. It was very worthwhile, and I hope to talk with him more in the future.





    SXSW report, Friday: Connections

    I'm attending SXSW Interactive on a last-minute transfer ticket, so I'm coming in with a little different aim. I'm looking at panels that further my jobs at the Statesman and as an adjunct at the University of Texas, but I also want to see some things that have nothing to do with either.

    Friday was all about connections. I actually only caught partials of two panels, but it was meeting other people where the day paid off. I started the day at RefreshAustin's meetup at BD Riley's at lunch time, and ran into two interesting folks:

    Peter Martinez is the founder of CloudShopper.com, but it was really three other interests that had us in intense conversation. As one of the leaders of RefreshMiami, they run a meetup that regularly draws 300 attendees. Like RefreshAustin, they try to be a hub for other tech groups in town, and now with a Knight grant they are really trying to extend that mission. Peter also taught a course on data visualization at the University of Miami, which of course I'm very interested in as I'm teaching a similar course at the University of Texas. But the most interesting connect was with TheLaunchPad, a entrepreneur group that helps college students take their ideas to the market. I figured Robert Quigley would want to talk to them in relation to his Mobile New App Design course he is teaching at UT.

    Jorge Gonzalez is a freelance motion designer that does video explanations, much like what I want to teach my students in my data visualization class. We had a great discussion on taking story ideas and finding concepts to help develop them into video explainers.

    Past that, I caught the Q&A for Bre Pettis's talk on MakerBot, and started the "Future of Crisis Communications" panel, but left to try and catch the Poynter eye-track study. Bad move, as it was full.

    Met and talked with one of the guys who helped make ProPublica's Fracking Song video explainer, which is my favorite video explainer ever, so it was a meet-the-rock-star moment, even though he was "second-rapper."

    Also spotted Bruce Sterling in the convention center. He typically has the closing remarks for the festival, but I haven't gotten that far in the schedule yet.

    Monday, January 21, 2013

    Military pharmacy costs and summing across spreadsheets

    I was thinking of a way to write about a story I helped with recently, Soaring cost of military drugs could hurt budget, but it wasn't really a technical masterpiece. Again, this is an example of a data-driven story where the best work was done in reporting.

    Our attempt (Jeremy Schwartz and I) to find out how much the military spent on pharmaceuticals proved quite difficult. There are at least three methods they use: The Defense Logistics Agency buys drugs from pharmaceutical companies for use in military hospitals and pharmacies. The Department of Defense health care system, Tricare Management Activity, provide prescription services through retail pharmacies, just like other health care pharmacies. Tricare also offers members prescriptions through mail order drugs.

    We were able to get pharmaceutical sales from DLA and Tricare, but not the mail order drugs. The records were difficult to work, as company names and drug names were not consistent. Cleaning up the data was more than adjust name ... the bulk came through research to find out what companies merged with others or were subsidiaries. That was just work. I created a new column in the spreadsheet and Jeremy and I each spent time doing research on company websites and other sources to find combinations as best we could.

    Both the Tricare and DLA worksheets have the
    same rows of companies in the same order,  as do the columns.
    Once Excel trick I did use was to total cells from multiple worksheets. This isn't all that rare for those who use Excel often, but it might be something a general reporter may not know how to do, so I thought I would explain it here.

    In my example, I have three worksheets in my spreadsheet file: DLA, Tricare, Combined. I had company totals by year in each column for both DLA and Tricare.

    On the Combined worksheet, I used the following formula to add the companies together:

    =sum(Tricare!B2+DLA!B2)

    You are using the values of the B2 cell from another worksheet by prepending it with the name of the worksheet and the exclamation point.

    You can then copy the formula to other cells, like any other formula.

    Why can't we add notes mode to SQL in MS Access?

    I had two stories I worked on run in the last 30 days where much of the work was done in the months before. It's at hard way to work ... to get pulled back into something and expect to answer questions about your analysis when you haven't looked at the data in weeks or months.

    To help with this, I sometimes keep really detailed notes on the steps I've taken when I analyze data. It can be time consuming and maddening, when you are redoing work after finding a mistake, but it is often those same notes that help you find the mistake.

    It would the REALLY helpful is Access would allow notes mode in SQL statements so users could notate their code. The ability to do this in MySQL workbench, along with the ability to execute highlighted code only, may cause me to eventually bail on Access, but I do really like the integration with Excel and even Word. I should explore more, but there is such an intellectual investment that has to be made to get comfortable with MySQL, I'm not sure I'll make that leap.

      © Blogger template 'A Click Apart' by Ourblogtemplates.com 2008

    Back to TOP