NYC Data #368: Shortening A/B Test Length with CUPED & CUPAC at Etsy, Developer Experience for AI Agents, Chatbot Regulation and Family Law, ADE-Bench, Squarespace Data Jobs,
Plus, Everyone in Seattle Hates AI?
Hi friends, I hope your Thanksgiving travel was smoother than mine; I ended up stuck in the Midwest for a few extra days! The end-of-year rush is on, but lots of interesting things to share with you this week.
As always, help me keep this space up-to-date: please send me posts, events, and job openings. If you know someone who might enjoy or benefit from this newsletter, please share it with them. [image credit: Adrienne Grunwald / New York Times]
Good Local Posts
Kelly McManus wrote about how Etsy uses pre-experiment data to shorten A/B test length. The general idea is that accounting for differences in the treatment/control audience based on previous information allows experimenters to reduce experimental noise, providing more precise effect estimates. Very interesting dive into CUPED and CUPAC.
Michael Waskom (author of Seaborn) and Rebecka Storm of Modal wrote about good developer experience for engineers and AI agents. This is the first work I’ve seen that talked not about making agents more capable, but about adjusting development patterns to make it easier for agents to complete tasks. The principles are easy to understand (the essay they link to on Locality of Behavior is worth a look as well).
I don’t spend much of my time reading law research, but “AI Companions and the Lessons of Family Law” by Clare Huntington really captured my attention (this article is a good summary). There are too many good quotes to choose just one but the general idea is that we should start with family law as a framework to think about rules and boundaries with AI chatbots, starting with principles of vulnerability, duty, and care. Long but really fascinating: reading this was truly a “science fiction come to life” moment for me.
Upcoming In-Person Events (new listings in bold)
12/9: Apache Spark in Production
12/10 - 12/11: The AI Summit New York
12/11: Applied AI in Finance and Banking
12/11 - 12/12: ODSC Agentic AI Workshop
Open Roles
Squarespace is hiring for a half-dozen data roles in NYC at the moment, including:
Squarespace is a terrific place to work: we host millions of websites and are one of the largest domain registrars in the world, which gives us a really unique view of the internet. Please send these along to anyone you know who might be interested!
Miscellany
‘Everyone in Seattle Hates AI’ by Jonathon Ready (which I saw via Trey Causey) has me thinking about the technical opinions we develop as a community, and how large corporations influence our opinions of tech, for good and ill. Trey says ‘Seattle might be the epicenter for anti-AI sentiment among major US cities’ and it makes me wonder what sentiments are more or less prevalent in the NYC tech community.
dbt is having an event next week where Benn Stancil will introduce ADE-bench, ‘the world’s first comprehensive benchmark for AI-driven analytics and data engineering’. I’m looking forward to it!
Thanks so much for being a subscriber. To see previous job listings (many of which are still open!) and blogs, check out the archive, which has emails from the tinyletter days. Feel free to forward this to anyone - they can subscribe here:
