NYC Data #384: Data Assets at Netflix, Ramp, The NBA, Visualizing SQL Performance, BERT & LLMs for Tagging, Nate Silver's Postmortem on FiveThirtyEight, The Pudding on Similes, NYC DOT
Also, lots of job postings!
Hi friends, Memorial Day weekend is finally here, but it looks like it’s going to be a rainy one 🌧️. I’m staying local and hoping that things dry up by Monday!
As always, help me keep this space up-to-date: please send me posts, events, and job openings. If you know someone who might enjoy or benefit from this newsletter, please share it with them. [image credit: Seth Hoffman]
Good Local Posts
Managing Data Assets at Netflix Scale was a really timely read for me. I’ve definitely dealt with pipelines failing due to permission/access issues when an engineer left my team. My teams have moved to project-based permissioning, but I hadn’t thought about the ‘Gravity’ concept before, or going backwards from the project for assessing issues.
In February I linked to a post from Datadog on visualizing Postgres EXPLAIN ANALYZE statements for debugging latency. This month, they published a followup on some new functionality to identify costly nodes in multi-join queries. This UX includes highlighting the SQL clauses that are driving cost, which is a pretty cool IMO.
Vicki Boykis used BERTopic and LLMs to tag her blog posts. Vicki includes not only a lot of technical detail, but a brief history of tags, which brought back lots of memories. Highly recommended! My teams have also found BERT to be better than LLMs in these basic text categorization tasks.
Upcoming In-Person Events (new listings in bold)
6/1 - 6/7: New York Tech Week
6/2: Search, Streams, and System Design Tradeoffs
6/2: Understanding and Advancing General Intelligence Through Games
6/9 - 6/10: Dash by Datadog
6/10: NYC Airflow Meetup at Astronomer HQ
6/17: AWS Summit NYC
6/22 - 6/26: UN Open Source Week
9/28 - 10/2: MLCon
Open Roles
There have been a number of major layoffs in the last few weeks. If you are hiring for a data role in the NYC area, please let me know and I’d be happy to post it here.
I’m hiring a number of roles at The New York Times, including:
Senior Analysts (many roles)
… and many more. Please apply through the website: feel free to let me know as well!
Outside of The Times, others are hiring as well:
Friends at Ramp are hiring Data Scientists.
Peloton is looking for a Data Scientist, Product Analytics.
NYC’s Department of Transportation is searching for a Data Engineer focusing on app-based deliveries and micromobility!
The NBA is looking for an Analytics Engineer.
WSJ Intelligence is looking for a Research Analyst.
Miscellany
A new Pudding interactive on similes, where they analyzed thousands of similes from literature. As fun as ____?
Nate Silver wrote a full postmortem on FiveThirtyEight and the relationship with Disney. Sad to see another old site I loved gone from the internet.
Thanks so much for being a subscriber. To see previous job listings (many of which are still open!) and blogs, check out the archive. Feel free to forward this to anyone - they can subscribe here:
