[{"content":"","date":"2026-05-28","externalUrl":null,"permalink":"/polimath/category/ai/","section":"Categories","summary":"","title":"AI","type":"categories"},{"content":"","date":"2026-05-28","externalUrl":null,"permalink":"/polimath/category/","section":"Categories","summary":"","title":"Categories","type":"categories"},{"content":"A while back I gave my first talk to a general audience about AI. The presentation went well, and it was generally very well received. Then came the Q\u0026amp;A.\nThe person who spoke first was nearly in tears. They went on at length about the evils of AI and how they would never use it. They said AI is taking jobs (true) and that all the jobs it takes can be done better by humans (not true, in many cases). The speaker used up the time scheduled for questions. They ended, less with a question and more with a challenge to rebut.\nI was caught off guard, but I was neither surprised nor offended. I listened patiently, or at least I tried to convey my sincere empathy.\nWhen given a chance to respond, I tried to validate their feelings and acknowledge that the very real uncertainty about AI\u0026rsquo;s current and future effects understandably evoke anxiety and even fear. I told the full audience that their concerns were important and I encouraged them to learn and think about those concerns. I did my best in the moment, but a more thoughtful response would be useful. This is the first part of my series of posts comprising that response.\nAI makes reasonable people concerned, anxious, and even afraid. Whether you have decided to support AI, fight AI, ignore AI, or haven\u0026rsquo;t decided what to do, these feelings are real and will interact with your response to AI.\nMuch has been said about the effects of AI on society; the type, size, and speed of these effects; and the philosophical, moral, and even religious implications. Much, much more consideration and discussion is essential. I am not taking on those topics here.\nMy thesis in this first part of the series is that companies must take into account the varying feelings of employees, customers, and society to succeed with the company\u0026rsquo;s AI strategy.\nAI can generate outsized business value # A recent MIT study said in July 2025,\nDespite the rush to integrate powerful new models, about 5% of AI pilot programs achieve rapid revenue acceleration; the vast majority stall, delivering little to no measurable impact on P\u0026amp;L. The research — based on 150 interviews with leaders, a survey of 350 employees, and an analysis of 300 public AI deployments — paints a clear divide between success stories and stalled projects.\nThe GenAI Divide: State of AI in Business 2025\nHowever, this is misleading about the value of AI in enterprise settings.\nLet\u0026rsquo;s compare this \u0026ldquo;failure\u0026rdquo; rate to those of other technologies that have been successful.\n85% of machine learning POCs are never deployed. 90% of Tableau dashboards aren\u0026rsquo;t used at all after six months. 90% of startups fail. 90% of drugs starting clinical trials never gain FDA approval. 95-98% of patents see no financial return. \u0026gt;\u0026gt;\u0026gt;95% of novels are never picked up for publication. Of those that are then self-published, \u0026gt;95% sell fewer than 1k copies. A 95% \u0026ldquo;failure\u0026rdquo; rate is common with some technologies that are nonetheless worth investing in. If a use case generates large returns and the implementation uses best practices to reduce the chances of failure, the ROI can redefine or create an industry.\nAI has already redefined and created industries. There are many failures, too, as the MIT study records. (FWIW, it\u0026rsquo;s a clear read about some good practices.) The potential upside, though, is clear.\nFeelings vary; many people are cautious # There are people without strong opinions about AI, but they are in the minority. A September 2025 Pew study shows, for example, that among U.S. adults,\n\u0026ldquo;50% say they\u0026rsquo;re more concerned than excited about the increased use of AI in daily life, up from 37% in 2021. 10% are more excited than concerned.\u0026rdquo; (40 points underwater) \u0026ldquo;In their own lives, about six-in-ten say they\u0026rsquo;d like more control over how AI is used.\u0026rdquo; (20 points underwater) \u0026ldquo;53% say AI will worsen people\u0026rsquo;s ability to think creatively, compared with 16% who say it will improve this.\u0026rdquo; (37 points underwater) Young adults are more likely to use AI (unsurprising), but are more pessimistic about its effects on society. It can be useful to think in terms of personas.\nAll-in: Positive about the potential for AI, and may use it regularly outside of work. Open-minded: Ready to learn, curious about what it means for them. Reluctant: Tentative, but reachable if treated with respect rather than solely as a cost to be reduced. Against: Very difficult to reach. Most people are a mix, but if your strategy addresses these four personas, you have a good chance of addressing those mixes of personas.\nFailure modes: how this can go wrong # There is tension between corporate goals and the fears of some employees and customers. There are many ways these feelings can reduce or even reverse gains from AI use.\nFailure modes by persona (just a few examples) # All-in # Can get frustrated with unnecessary restrictions on the use of AI. If they are used to using the latest models at home and the company offers substantially dumber tools, they will use personal AI accounts, which can lead to private data being leaked and even used as training data in public models.\nOpen-minded # May feel unnecesary pressure to learn AI tools on their own time in order to be able to do their current-but-being-redefined jobs. If their manager says, \u0026ldquo;We\u0026rsquo;re going to reduce drudgery!\u0026rdquo;\nthen they might ask, \u0026ldquo;Am I going to be doing more creative work, or just being replaced? Am I even capable of doing this new work they expect of me?\u0026rdquo;\nReluctant # Redefining their jobs means they are being asked to do jobs they did not apply for, may not like, and may not be capable of. Providing training data for an AI may echo the negative psychological effects of offshoring.\nAt the same time that this American engineer was training his foreign replacement, the CEO of his company was publicly complaining to Washington policymakers about a shortage of U.S. engineers.\nThe Offshoring of Innovation, The Economic Policy Institute Against # This persona may have significant trouble adjusting to a culture that is embracing AI. Not only is it possible that they may not use AI tools effectively, but they may also decrease the effectiveness of others and potentially, either intentionally or unintentionally, reduce the success of AI applications. Feelings about AI can be very strong if they feel it is an existential threat.\nHappy path: how this can go well # There are already some industry good practices for how to cultivate, rather than impose, a work culture that supports the effective use of AI.\nTransparent and frequent communication # Proactively share why AI is being introduced, what roles it will play, and how it may impact jobs, responsibilities, and workflows. Address both benefits and risks, and clarify what AI can and cannot do within the organization, including specific policies on data privacy and job changes. Offer multiple communication channels, such as meetings, written summaries, and Q\u0026amp;A sessions. These can align with diverse information-processing preferences among employees. Inclusive decision-making and employee voice # Involve employees in AI adoption decisions through pilot projects, feedback surveys, and focus groups. Create safe spaces for employees to ask questions, share concerns, and express excitement or skepticism, ensuring all perspectives are valued and heard. Strong, supportive leadership # Leaders can model openness and curiosity about AI, sharing their own learning process to foster psychological safety for experimentation and adaptation. Appoint steering committees or leader champions to guide ethical adoption, answer employee questions, and maintain organizational alignment. Personalized training and upskilling # Provide reskilling and upskilling opportunities, supporting employees as they transition to working alongside AI or into new roles. Tailor training to employee experience levels. This can be advanced training for tech-savvy teams, hands-on sessions for skeptics or less experienced staff, and ongoing development for everyone. Emphasize how AI can help reduce repetitive work and unblock employees so they can focus on higher-value tasks and career growth. Well-being and purpose # Regularly check in on employee well-being and tech-related stress, and make adjustments as needed. Align AI adoption with the organization\u0026rsquo;s purpose and values, ensuring changes do not undermine trust or culture. Implementing these practices ensures organizations are not only technologically innovative but also empathetic, ethical, and supportive. This can help folks with a range of attitudes toward AI feel more informed, involved, and secure. Enabling experimentation # The recent MIT study mentioned before reported that one high-return method for generating financial impact with AI is as simple as providing access to a chatbot, such as ChatGPT.\n[I]individuals can successfully cross the GenAI divide when given access to flexible, responsive tools. The organizations that recognize this pattern and build on it represent the future of enterprise AI adoption.\nThe GenAI Divide: State of AI in Business 2025 Taking a bottom-up approach is a strong strategy because:\nIt fosters creativity. (Isn\u0026rsquo;t that what we\u0026rsquo;re telling people their competitive advantage is relative to AI?) It lets people learn when the stakes are low and they\u0026rsquo;re not being watched. It encourages curiosity, continual learning, and collaboration. It can help identify uses of AI that can eventually work at the team or enterprise level. It gets people on board with the utility (and even fun) of using AI. Be a leader, not just a manager # Leaders who grok the difference between management and leadership know that feelings have clear effects on the bottom line. Recognizing that feelings about AI are strong and then incorporating them into AI strategy is ignored at the company\u0026rsquo;s peril. Just as important, when our world is navigating a change likened to the Industrial Revolution at 21st Century speed, treating your team as humans, not resources, fosters the change you want to see in the world.\n","date":"2026-05-28","externalUrl":null,"permalink":"/polimath/corporate-empathy-about-ai/","section":"Posts","summary":"A while back I gave my first talk to a general audience about AI. The presentation went well, and it was generally very well received. Then came the Q\u0026A.\nThe person who spoke first was nearly in tears. They went on at length about the evils of AI and how they would never use it. They said AI is taking jobs (true) and that all the jobs it takes can be done better by humans (not true, in many cases). The speaker used up the time scheduled for questions. They ended, less with a question and more with a challenge to rebut.\n","title":"Corporate empathy about AI","type":"post"},{"content":"","date":"2026-05-28","externalUrl":null,"permalink":"/polimath/category/data-science/","section":"Categories","summary":"","title":"Data Science","type":"categories"},{"content":"","date":"2026-05-28","externalUrl":null,"permalink":"/polimath/tag/ethical-ai/","section":"Tags","summary":"","title":"Ethical AI","type":"tags"},{"content":"","date":"2026-05-28","externalUrl":null,"permalink":"/polimath/category/leadership/","section":"Categories","summary":"","title":"Leadership","type":"categories"},{"content":"","date":"2026-05-28","externalUrl":null,"permalink":"/polimath/","section":"Polimath","summary":"","title":"Polimath","type":"page"},{"content":"","date":"2026-05-28","externalUrl":null,"permalink":"/polimath/post/","section":"Posts","summary":"","title":"Posts","type":"post"},{"content":"","date":"2026-05-28","externalUrl":null,"permalink":"/polimath/category/strategy/","section":"Categories","summary":"","title":"Strategy","type":"categories"},{"content":"","date":"2026-05-28","externalUrl":null,"permalink":"/polimath/tag/","section":"Tags","summary":"","title":"Tags","type":"tags"},{"content":" Education in the age of AI should teach everything. # It\u0026rsquo;s easy to fall into the trap of thinking AI handles breadth, so school should narrow. Drop the languages, skip the history, double down on STEM, add a \u0026ldquo;prompt engineering\u0026rdquo; elective. That gets it backwards.\nThe person who gets the most out of AI is the one with the widest education, not the deepest.\nHere\u0026rsquo;s why:\nAI is good at anything, not everything\u0026hellip; \u0026hellip;and the connection space is too large to brute force\u0026hellip; \u0026hellip;and humans intuit strong questions\u0026hellip; \u0026hellip;and specialization is becoming management\u0026hellip; \u0026hellip;and humans intuit which questions matter\u0026hellip; \u0026hellip;and diverse thinking beats \u0026ldquo;best\u0026rdquo; thinking.\nAI is good at anything, not everything # Pick any single domain - tax law, organic synthesis, sonnet writing, kubernetes - and a frontier model will perform somewhere between a competent practitioner and an expert. Pick the intersection of two unrelated domains and the quality drops sharply. \u0026ldquo;Write me a sonnet about reconciling EBITDA add-backs in a SaaS LBO model\u0026rdquo; is technically possible and noticeably worse than either skill alone.\nThe breadth is real. The depth at any arbitrary intersection is not.\nThe connection space is too large to brute force # The relevant scale is not what one human carries. It is what the AI was trained on. English Wikipedia: 7 million articles. Pairs exceed 25 trillion. Triples exceed 60 quintillion. Frontier training sets are much larger than 7 million. No model precomputes those connections, and none can retrieve the relevant one without a prompt that names it.\n[insight(a, b) for a, b in itertools.combinations(all_human_knowledge, 2)] is not a strategy. The cross-domain insight has to be requested. Someone has to suspect it might exist.\nHumans intuit strong questions # Ask a vague question, get a vague answer. Ask a precise question with sharp vocabulary and explicit constraints, get something useful. The bottleneck on AI output is usually the input.\nA student trained only in CS can ask CS questions well. A student also trained in history, ethics, music, and biology can ask questions that touch all of those. That is where AI output looks superhuman, because the human did the hard part.\nSpecialization is becoming management # A human being should be able to change a diaper, plan an invasion, butcher a hog, conn a ship, design a building, write a sonnet, balance accounts, build a wall, set a bone, comfort the dying, take orders, give orders, cooperate, act alone, solve equations, analyze a new problem, pitch manure, program a computer, cook a tasty meal, fight efficiently, die gallantly. Specialization is for insects.\nRobert A. Heinlein, Time Enough for Love The job of a senior IC in 2026 looks a lot like the job of a manager in 2016: set context, decompose the problem, evaluate the output, push back when it drifts. Computer scientists are rediscovering management theory. Huh.\nMicrosoft\u0026rsquo;s December 2025 New Future of Work Report puts it directly: workers are \u0026ldquo;shifting from merely doing work to guiding, critiquing, and improving the work of AI.\u0026rdquo; That is a job description for a manager.\nManaging five AI agents is not a different skill from managing five junior engineers. It is the same skill applied at higher throughput. Management has always rewarded breadth.\nHumans intuit which questions matter # Formulating a question is one skill. Choosing which question to ask at all is a different one, and harder. It is the difference between a research assistant and a principal investigator.\nAI does shift the calculus. When pursuing a question takes minutes instead of months, more speculative chases become affordable, and the bar for \u0026ldquo;worth running\u0026rdquo; drops. The intuition for which speculations to run still has to come from somewhere.\nThat somewhere is pattern recognition across domains: something a generalist accumulates over years, and a specialist often does not.\nDiverse thinking beats \u0026ldquo;best\u0026rdquo; thinking # Hong and Page (PNAS, 2004) proved that under reasonable conditions, a group of cognitively diverse problem-solvers outperforms a group of the highest-ability problem-solvers on hard problems. The intuition: diverse heuristics cover more of the search space than redundant strong ones do.\nML system designers already commit to this. Amazon\u0026rsquo;s homepage runs multiple recommenders side by side - item-to-item collaborative filtering, browsing history, frequently-bought-together, and more, because no single A/B test winner algorithm beats the ensemble. The people who could engineer one optimal model build many imperfect ones instead.\nA human plus several AIs is the same play. The human contributes the heuristic the AIs do not have. That contribution scales with breadth and collapses without it.\nEducate the whole person. In the age of AI, the liberal arts are not a luxury - they are the training ground for a role that endures. # We homeschooled our kids on exactly the bet that breadth would compound and narrowness would not. So far, it has.\n","date":"2026-04-29","externalUrl":null,"permalink":"/polimath/raising-an-expert-at-using-ai/","section":"Posts","summary":"Education in the age of AI should teach everything. # It’s easy to fall into the trap of thinking AI handles breadth, so school should narrow. Drop the languages, skip the history, double down on STEM, add a “prompt engineering” elective. That gets it backwards.\n","title":"Raising an Expert at Using AI","type":"post"},{"content":"","date":"2026-04-29","externalUrl":null,"permalink":"/polimath/tag/teaching/","section":"Tags","summary":"","title":"Teaching","type":"tags"},{"content":"","date":"2026-01-07","externalUrl":null,"permalink":"/polimath/tag/good-practices/","section":"Tags","summary":"","title":"Good Practices","type":"tags"},{"content":"There are only two hard things in Computer Science: cache invalidation and naming things.\nPhil Karlton Bad names create confusion. Good names:\nProvide clarity to consumers about what to find inside by telling us why we have the named thing. Provide guidance to producers about what should and should not go inside, and tell us how we create the named thing. Are easy to use: concise, memorable, easy-to-spell, not random or disconnected from how and why we have the named thing. Avoid \u0026ldquo;collisions\u0026rdquo; = repeating a name of something else that is not distinguishable from context. This is not a complete guide to naming, but rather a compilation of good practices1 that generalize what I have found useful. We\u0026rsquo;ll discuss:\nCoding Projects Stories: presentations, reports, individual slides and sections Email subjects Calendar events Children Coding # variables, methods, file names snake_case: compatible across languages and file systems lower case: easy to remember no hyphens: hyphens mean subtraction in mathematical languages like R no spaces: spaces in file names complicate file handling methods are verbs CONSTANT: CAPITALIZED, singular or plural ClassDefinition: TitleCase, singular git-repository: lower-case-and-hyphenated Make names descriptive but somewhat concise. The length of a variable name should be proportional to the scope of the variable to avoid collisions and hint at where it is defined. Examples of good names: i for the index of a very short code block customer_id within a class or function bridge_customer_id when assigned by the \u0026ldquo;Bridge\u0026rdquo; team CUSTOMER_ID_ENCRYPTION_SALT_PREFIX for use across a wide scope Projects # There are generally two consumers of project names:\nWorkers: those working on the project Stakeholders: leadership and everyone else This suggests two-part names: \u0026ldquo;{Method} {Purpose}\u0026rdquo; or \u0026ldquo;{Purpose} {Method}\u0026rdquo;\nPurpose or Value: Tells stakeholders how to use or why they should care. Avoids collisions among workers\u0026rsquo; projects. Method or Tool: Stakeholders avoid collisions (\u0026ldquo;This is the {purpose} that\u0026rsquo;s different because it uses {method}\u0026rdquo;). Insufficient for workers, who use the same method on other projects, so we add the purpose. Examples: AI Pricing Model\nStories # Don\u0026rsquo;t bury the lede: tell the story in the title (name). Start with the conclusion. Don\u0026rsquo;t wait until the end2 to reveal the answer to the question. If When you lose the audience\u0026rsquo;s attention, it\u0026rsquo;s too late. How often do you finish your presentation early after covering all of your planned points? (Not often.) How often does every consumer read to the end? (Not often.)\nToo often, we tell the history of how we reached a conclusion. Your audience doesn\u0026rsquo;t care how you spent your week. The answer is the most important part. After giving the answer, make a concise, reasoned argument as to why they should believe the answer; this will have selected parts of the history but not every blind alley. This isn\u0026rsquo;t an itemized time sheet.\nExamples:\nBad presentation/report title: \u0026ldquo;Q4 Performance\u0026rdquo; Good presentation/report title: \u0026ldquo;2025 Q4 Performance: Continued Growth\u0026rdquo; Putting the full time period (including year) at the beginning means that default file sorting (alphabetical) puts files in chronological order.\nBad slide/section title: \u0026ldquo;Proposed Organizational Structure\u0026rdquo; Good slide/section title: \u0026ldquo;Centralized Services Reduce Unnecessary Redundancy\u0026rdquo; The good title has a verb that connects the how to the why.\nEmail subjects # Subject lines (usually) should tell the story. Two notes:\nIf the contents are sensitive, avoiding the answer (or even the specific question) may make it easier for recipients to avoid leaking when sharing their screen or when someone stops by their desk. Remember that emails often are forwarded, so subjects should be descriptive and avoid collisions for people one step removed from the recipients, if possible. Calendar events # Conciseness is necessary because only the first part of the event title is visible in a one-week view without opening the event.\nDon\u0026rsquo;t write \u0026ldquo;meeting\u0026rdquo;. We know it\u0026rsquo;s a meeting. Don\u0026rsquo;t write \u0026ldquo;Discussion\u0026rdquo; or \u0026ldquo;Presentation\u0026rdquo;. We know. Focus on the goal of the meeting. Bad: \u0026ldquo;{sender} - {recipient}\u0026rdquo;. Looks like bad news. Good: \u0026ldquo;1-1: {sender} / {recipient}\u0026rdquo;. Looks routine. Better: \u0026ldquo;1-1: {recipient} / {sender}\u0026rdquo;. (Recipient first.) When the title gets cut off, you can still see who you\u0026rsquo;re meeting with. Bad: \u0026ldquo;Org update\u0026rdquo;. Looks like terrible news. Good: \u0026ldquo;Org update\u0026rdquo; (the same?) but let them know before sending the invitation that it\u0026rsquo;s not bad news for them. In general, avoid sending invitations that could be interpreted as bad news (such as the recipient being fired, you quitting, etc.)\nBonus: Give context inside the invitation\u0026rsquo;s description: agenda, suggested/expected preparation, goals, in-scope/out-of-scope. The invitation, including description, is the \u0026ldquo;name\u0026rdquo; of the meeting.\nOut-of-Office \u0026ldquo;events\u0026rdquo; # Bad: \u0026ldquo;Out of Office\u0026rdquo;. Wait, who is out of the office? The recipient? That\u0026rsquo;s what it will look like to them. Good: \u0026ldquo;{sender} OOO Jan 12-20\u0026rdquo;. Tells the whole story concisely. Bonus good practices:\nMake them \u0026ldquo;all-day\u0026rdquo; events, not an event repeated every day. This makes it appear as a single bar across the top of the calendar. Don\u0026rsquo;t use the event you send to others as a way to block off your time; if it blocks off the day, then their calendar will be blocked off! To block off the hours, create a separate event that you don\u0026rsquo;t send out, 12 am to 12 am the next day, that is repeated. That way, others will know you\u0026rsquo;re not available. Children # Call them whatever you like! However, here are some considerations.\nThe child will have the name for their entire life. Try to avoid making therapy inevitable. You might aim for something that: Someone reading it will know how to pronounce it. Someone hearing it will have a good chance of knowing how to spell it. If you get \u0026ldquo;creative\u0026rdquo; with the spelling, they will have to spell their name and help others pronounce it. Forever. They will grow up. You can call them \u0026ldquo;Stevie\u0026rdquo; when they are young, but you might want to put \u0026ldquo;Stephen\u0026rdquo; on their birth certificate. Names from today\u0026rsquo;s culture will reveal their age. This can lead to ageism 50 years from now. When you hear about a \u0026ldquo;Karen\u0026rdquo; or a \u0026ldquo;Gary\u0026rdquo;, do you think Gen Z? Probably not. Family names can be deeply meaningful \u0026ndash; great! Some should be avoided Example: my dear grandfather, Adolphus. These practices aren\u0026rsquo;t intended to be dogma, but now if you choose to go a different way, you\u0026rsquo;ll have some idea about the tradeoffs.\nThe More You Know \u0026ldquo;Best\u0026rdquo; practices are debatable. \u0026ldquo;Good\u0026rdquo; practices avoid known pitfalls and needless debates.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nI\u0026rsquo;m looking at you, academic researchers.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"2026-01-07","externalUrl":null,"permalink":"/polimath/good-practices-naming-things/","section":"Posts","summary":"There are only two hard things in Computer Science: cache invalidation and naming things.\nPhil Karlton Bad names create confusion. Good names:\nProvide clarity to consumers about what to find inside by telling us why we have the named thing. Provide guidance to producers about what should and should not go inside, and tell us how we create the named thing. Are easy to use: concise, memorable, easy-to-spell, not random or disconnected from how and why we have the named thing. Avoid “collisions” = repeating a name of something else that is not distinguishable from context. This is not a complete guide to naming, but rather a compilation of good practices1 that generalize what I have found useful. We’ll discuss:\n","title":"Good practices: Naming Things","type":"post"},{"content":"The easiest way to get something done is to get someone else to do it.\nStephen Haptonstahl Thanksgiving dinner # Dad is about to put the turkey in the oven. There\u0026rsquo;s a lot of food yet to prepare. Three guests wander into the kitchen and ask how they can help. The last thing Dad needs is a distraction!\nDad figures it will take him an hour to finish preparing the meal. The guests don\u0026rsquo;t know the kitchen, so they can only do what is needed half as fast, and Dad won\u0026rsquo;t be able to do anything at all if he\u0026rsquo;s showing them what to do.\nDad\u0026rsquo;s choices:\nDo it himself. 60 minutes. Step away from the knives and focus entirely on showing them what to do. The guests do the work. 40 minutes. Bonus: Dad gets to tell sea stories while they prepare the feast, making it a social activity.\nPlatforms and User-Generated Content # Jack Musckerberg wants to start a website where they can sell ads. The problem: He needs content to attract visitors, and he needs new content continually to keep them coming.\nJack\u0026rsquo;s choice:\nHire content creators to generate content. Marginal cost per page of content: $25. Create a platform where the users themselves post pictures of cats. Marginal cost per page of content: \u0026lt; $0.01. Bonus: Safe Harbor (17 U.S. Code § 512) means he\u0026rsquo;s less liable for some problematic content.\nSoftware and plugins/extensions # The new application FaceHugger is a tool for aliens using machine learning models. ML moves fast, and it\u0026rsquo;s difficult to keep the software up-to-date with the latest algorithms.\nThe proprietor from another world has to choose:\nBring the entire hive to Earth to implement new models. Cost per model: One meal for the alien modeler. Make the software support plugins and give credit to hooman contributors. Cost per model: \u0026lt; $0.01. Moving Mt. Honeycrisp # Shia is interviewing at Aleph, a startup in search. Aleph hires strictly based on IQ, so their interview process asks some interesting questions.\n\u0026ldquo;How would you move Mt. Honeycrisp?\u0026rdquo;\n(Two of) Shia\u0026rsquo;s choices:\nBulldozers. Start a religion, where the highest ritual is a pilgrimage to Mt. Honeycrisp. Adherents are to take a piece of the mountain home with them. Bonus: Shia doesn\u0026rsquo;t have to worry about where to put the mountain.\nAssorted sorting # Sally is applying for a job as a machine learning engineer. Having met with the charismatic and empathetic hiring manager, the next step is the Python test. The examiner directs, \u0026ldquo;Write code to sort this DataFrame on the second column.\u0026rdquo;\nShe could:\nImplement a sorting algorithm from scratch. This is a coding test, right? She can show off her familiarity with bubble sort. Use a provided library: df_sorted = df.sort_values(by=1). Bonuses:\nIt\u0026rsquo;s more Pythonic. Total of (development time) + (runtime) is guaranteed to be faster, and the interview is only an hour. There will be fewer bugs and better handling of edge cases. But wait! Doesn\u0026rsquo;t that mean someone else is doing more work? Not if things are set up right.\nData science team collaboration # Chris and Barinder are experimenting with ML models. Each model has many similar, complicated parts (data splits without leakage, training, tuning, calibration, testing, etc.)\nThey could:\nWork separately, each creating their own code tuning and calibrating the models. Split the work, Chris building a generic tuning capability and Barinder building a generic calibration capability. Even with the extra work of generalizing the code, there will be more time to refine, test, optimize, and harden the code. Bonuses include:\nGiven enough eyeballs, all bugs shallow. Collaboration is part of the fun of being on a team. ","date":"2025-11-08","externalUrl":null,"permalink":"/polimath/get-someone-else-to-do-it/","section":"Posts","summary":"The easiest way to get something done is to get someone else to do it.\nStephen Haptonstahl Thanksgiving dinner # Dad is about to put the turkey in the oven. There’s a lot of food yet to prepare. Three guests wander into the kitchen and ask how they can help. The last thing Dad needs is a distraction!\n","title":"Get someone else to do it","type":"post"},{"content":"","date":"2025-08-01","externalUrl":null,"permalink":"/polimath/category/cake/","section":"Categories","summary":"","title":"Cake","type":"categories"},{"content":"I make birthday cakes for each of my boys. I don\u0026rsquo;t use fondant \u0026ndash; I strive for \u0026ldquo;tasty\u0026rdquo; \u0026ndash; so it can get a bit crazy.\nAnder has always been a fan of space. We\u0026rsquo;ve watched this IMAX movie about 6.02⋅10236.02\\cdot10^{23}6.02⋅1023 times.\nIMAX Space Station trailer This year, he asked for a two-layer, chocolate, NASA cake. Can do! Here\u0026rsquo;s my reference art.\nNo problem. I chose a chocolate fudge mix and whipped up a couple of layers.\nIt\u0026rsquo;s actually harder to frost a blue circle than it would seem.\nLet the details begin!\nHappy birthday, Ander!\n","date":"2025-08-01","externalUrl":null,"permalink":"/polimath/cake-ander-18-nasa/","section":"Posts","summary":"I make birthday cakes for each of my boys. I don’t use fondant – I strive for “tasty” – so it can get a bit crazy.\nAnder has always been a fan of space. We’ve watched this IMAX movie about 6.02⋅10236.02\\cdot10^{23}6.02⋅1023 times.\n","title":"Cake Ander-18: NASA","type":"post"},{"content":"I make birthday cakes for each of my boys. I don\u0026rsquo;t use fondant \u0026ndash; I strive for \u0026ldquo;tasty\u0026rdquo; \u0026ndash; so it can get a bit crazy.\nThis year, Artemis was away at school learning to become a carpenter at North Bennet Street School in Boston, the oldest trade school in the country. Wow, did he learn a lot! With him hundreds of miles away, I couldn\u0026rsquo;t make him a cake for his mid-winter birthday, so I did something almost as good: a half-birthday cake.\nI decided to surprise him with a cake in the form of a power tool he\u0026rsquo;d been wanting: a SawStop table saw. If you haven\u0026rsquo;t heard of these, you\u0026rsquo;ve missed out. These saws have sensors that detect when a finger or other meat touches the blade, and then stops the blade so fast that the only injury is a nick. It\u0026rsquo;s worth watching this six-second demo with a hot dog:\nSawStop hot dog demo Amazing! What better way to celebrate a half-birthday than with something designed to keep you from having half a finger?\nThe saw blade was an interesting challenge. I went with a stroopwafel\u0026hellip;mmm, caramel goodness. A chocolate bar played the part of a board being hewn in twain, and voila!\n","date":"2025-07-14","externalUrl":null,"permalink":"/polimath/cake-artemis-19-5-sawstop/","section":"Posts","summary":"I make birthday cakes for each of my boys. I don’t use fondant – I strive for “tasty” – so it can get a bit crazy.\nThis year, Artemis was away at school learning to become a carpenter at North Bennet Street School in Boston, the oldest trade school in the country. Wow, did he learn a lot! With him hundreds of miles away, I couldn’t make him a cake for his mid-winter birthday, so I did something almost as good: a half-birthday cake.\n","title":"Cake Artemis-19.5: SawStop","type":"post"},{"content":" tl;dr # North star: Happier teams deliver more. How I sail that course: Remove demotivating emotional shielding, using empathy and vulnerability to create a safe space. Motivate by providing purpose, autonomy, and the chance to develop mastery. Don\u0026rsquo;t control: trust. Management is allocating resources. Leadership is influencing people toward goals.\n- Stephen Haptonstahl\nI\u0026rsquo;ve led teams.\nData teams: AI engineers, machine learning engineers, data scientists, data engineers, labelers, business analysts, data analysts, statisticians. Classrooms: from high school students to graduate students. Naval petty officers and seamen: submarine hunters, air controllers, firefighters. How do I motivate people toward goals? # Start with having clear goals.1 However, goals don\u0026rsquo;t motivate.\nMotivation is the wind that gets the team moving. Goals are the keel that keep the team pointed toward delivering value.\n- Stephen Haptonstahl\nWhat motivates? I focus on happiness for two reasons:\nHappy workers are more motivated. They are also less likely to leave, achieve more, take less sick leave, and help each other more. I like working with happy people. The beatings will continue until morale improves.\n- Old military saying\nWhat about more money? That seems easier, and it works, right? Sometimes, sometimes not.\nExample: Performance bonuses generate higher performance for simple, repetitive, mindless tasks like data entry. Counterexample: Children who enjoy drawing for fun may lose interest if they start receiving rewards for it, as the activity shifts from play to work. Even when it works, money can have limits as a motivator. Past about $105k/year, higher pay doesn\u0026rsquo;t increase a worker\u0026rsquo;s happiness.2 So what does?\nRemove demotivators with empathy and vulnerability # I lead with empathy. Your team will be more likely to let themselves be vulnerable if you do.\nWhen I had a major illness a few years ago, I told my team about it, that I was scared. I didn\u0026rsquo;t cast the future as dismal, but I was honest (without oversharing) about my uncertain prognosis. I shared my concerns about how it would affect my work and career. I told them I was confident in my leads and the whole team, that they would step up while I recovered.\nSeveral team members thanked me privately for being open. Some shared that they or someone close to them had been through a similar experience. My team did step up, knowing I believed in and counted on them.\nMore than once I\u0026rsquo;ve had a team member break down in tears during a weekly one-on-one because of a personal situation. I\u0026rsquo;m glad they felt comfortable enough to open up to me.\nEmpathy leads to a happier team and better performance\nFeeling safe at work has well-known performance advantages.\nIt\u0026rsquo;s one thing to say, \u0026ldquo;We don\u0026rsquo;t focus on blame, we focus on preventing the next problem.\u0026rdquo; That only works if your team feels safe owning their mistakes. When reorganization hits, those not directly affected are often afraid, not only for their own jobs, but also for their development and promotion. I recently left a job I loved and where I was succeeding because leadership left me in limbo for a month after a reorg was announced. By the time they said I and my team were valued, I had already received a better offer elsewhere. Add motivators: Purpose, Autonomy, and Mastery # Purpose # Having a purpose that aligns with your values is deeply satisfying. Here are some jobs I\u0026rsquo;ve had, and the stated purpose of each organization.\nUS Navy: Defend freedom, preserve economic prosperity, and keep the seas open and free. National Public Radio: Create a more informed public. Revecore: Help hospitals thrive so they can continue to serve their communities. Am I ready to step up and give high effort for those? No doubt.\nAdmittedly, an organization or culture can manipulate those in some professions (teaching, child care, nursing) using vocational awe by citing a higher purpose. I try to avoid setting that tone, and sometimes I\u0026rsquo;ll point out that \u0026ldquo;There are almost no emergencies in data science.\u0026rdquo;3\nAutonomy # I hire professionals.4 Professionals have knowledge, skills, and experience built up over years. It is wasteful and insulting to tell them how to get something done.\nA supervisor can do even better by asking teams to choose what to work on, or at least provide input; this is better management of their own time and better leadership by motivating the team.\nA good practice is to set requirements and leave the solutions to the experts.\nMotivating: Maximize revenue by improving the recommendations while not reducing margin, not increasing the budget more than 10%, and maintaining security. You tell me how to do it. Demotivating: Improve recommendations by adding these features, using this algorithm, and using this architecture. I\u0026rsquo;ll tell you what to work on first. FunFact: Most workers, professional or not, prefer to have some autonomy in their jobs.\nMastery # Professionals of all stripes have specialized skills. In data science and other data fields, those skills require constant learning and the chance to apply them.\nSome specific ways to encourage developing mastery:\nSupport training and conferences with time, money, and inclusion in evaluations of performance. Share lessons learned with each other. Ex: AI is changing how we work. Each week, each member of my team is invited to update a Confluence page with a very brief lesson they\u0026rsquo;ve learned about trying to use AI in their workflow. Set an example. When I present a possible solution that I just learned, I mention that I just learned it. It may help them see that I keep working at improving my technical skills. Hold lunch-and-learns with folks across the organization. Team members learn more by preparing to present, and they improve their presentation skills. Ask for help. If you create a safe space and then admit you don\u0026rsquo;t have all the answers but want to learn. Assign tasks that (a) leverage existing skills AND (b) require a new skill. Consider this: Your team likely won\u0026rsquo;t retire from your current organization. They\u0026rsquo;re going to be on the job market at some point. Are they burnishing the skills that will help them find that next job? If so, most will feel they are growing, safer, and will be less likely to leave. I have left jobs before because they focused too much on developing skills that wouldn\u0026rsquo;t transfer to other roles.\nMore on data team goal definition in future posts.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nIn 2010, \u0026ldquo;enough\u0026rdquo; was ~$75k/year in the US (Daniel Kahneman and Angus Deaton, 2010). In 2025, that\u0026rsquo;s ~$105k/year.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nI\u0026rsquo;ve dealt with real emergencies, like an engine room oil fire or recovering the bodies of friends who died at sea. Civilian data science doesn\u0026rsquo;t usually justify much cortisol.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nMore on \u0026ldquo;professionals\u0026rdquo; in future posts.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"2025-07-13","externalUrl":null,"permalink":"/polimath/how-i-lead-data-teams-principles/","section":"Posts","summary":"tl;dr # North star: Happier teams deliver more. How I sail that course: Remove demotivating emotional shielding, using empathy and vulnerability to create a safe space. Motivate by providing purpose, autonomy, and the chance to develop mastery. Don’t control: trust. Management is allocating resources. Leadership is influencing people toward goals.\n","title":"How I lead (data) teams: principles","type":"post"},{"content":"I make birthday cakes for each of my boys. I don\u0026rsquo;t use fondant \u0026ndash; I strive for \u0026ldquo;tasty\u0026rdquo; \u0026ndash; so it can get a bit crazy.\nAnder turns 16 # This was around the time The Super Mario Bros. Movie had come out and Ander had shifted from MK8DX (that\u0026rsquo;s Mario Kart 8 Deluxe on the Nintendo Switch, for you non-gamers) aficionado to ranked record-holder. Things were getting real. He asked for Mario, in chocolate, as usual.\nThis seemed ambitious, not so much for the physical construction but for the detail in shape and decoration. Here\u0026rsquo;s our beloved hero.\nI, not one to refuse help, found a Mario cake pan. (Of COURSE there\u0026rsquo;s a Mario cake pan.)\nNot only was there a pan, but it included paint-by-numbers instructions that even specified the order. Seems like this would be straightforward.\nI buttered the mold thoroughly to ensure the cake came out in one piece, and that almost worked, with only a couple of small pieces having to be glued with frosting into place.\nAfter that, the main decoration came together nicely.\nFinally, some personalization completed another cake. Ander loved it!\n","date":"2025-07-04","externalUrl":null,"permalink":"/polimath/cake-ander-16-mario/","section":"Posts","summary":"I make birthday cakes for each of my boys. I don’t use fondant – I strive for “tasty” – so it can get a bit crazy.\nAnder turns 16 # This was around the time The Super Mario Bros. Movie had come out and Ander had shifted from MK8DX (that’s Mario Kart 8 Deluxe on the Nintendo Switch, for you non-gamers) aficionado to ranked record-holder. Things were getting real. He asked for Mario, in chocolate, as usual.\n","title":"Cake Ander-16: Mario","type":"post"},{"content":"","date":"2025-07-04","externalUrl":null,"permalink":"/polimath/tag/mario/","section":"Tags","summary":"","title":"Mario","type":"tags"},{"content":"Two common uses for statistical (or machine learning) models are:\nSomething happened. Why? Something will happen. What? Spoiler Alert # When reporting results, tell your audience how certain you are. This is essential context. \u0026ldquo;Why?\u0026rdquo;: When presenting parameters (drivers, slopes, causal effects, and the like) use something like a confidence interval, Bayesian credible interval, or bootstrapping the data. \u0026ldquo;What?\u0026rdquo;: When presenting predictions, use a conformal prediction interval, Bayesian posterior predictive interval, or the like. Forecasts must use these. \u0026ldquo;Why?\u0026rdquo; = inference about drivers # When we ask \u0026ldquo;why?\u0026rdquo; we are trying to make an inference about how the data was generated. Specifically, inferences about the parameters of the model, which can be thought of as measures of the relationships between features and the target.\nFor example, if we\u0026rsquo;re using linear regression to fit a line to xxx and yyy, our model looks like:\ny=slope⋅x+intercept+error y=\\text{slope}\\cdot x + \\text{intercept} + \\text{error} y=slope⋅x+intercept+errorThe parameters of this model are the slope, intercept, and standard deviation of the error. In many cases, we\u0026rsquo;re interested in how xxx is related to yyy, which means the interesting parameter is the slope.\nFor example, we might use a dataset to estimate the parameters and find:\ny=2.6x+5.7+N(0,0.52) y=2.6x+5.7+\\mathcal{N}(0,0.52) y=2.6x+5.7+N(0,0.52)So, for every additional xxx we get 2.6 more yyy. How sure are we that it\u0026rsquo;s 2.6? Is it 2.6±0.12.6\\pm 0.12.6±0.1 or 2.6±32.6\\pm 32.6±3?\nThe frequentist way to describe this uncertainty is to use a confidence interval. You can also use a Bayesian credible interval, a bootstrap interval generated by resampling the data, or other methods. Each has a somewhat different interpretation, but all are ways to convey how certain we are about how much more yyy is associated with one more xxx.\nYou can also perform a hypothesis test to estimate how sure you are that the slope is measurably different from zero (\u0026ldquo;reject the null hypothesis\u0026rdquo;). A Bayesian alternative is to measure how likely it is that the slope is positive (counting the fraction of posterior draws greater than zero).\nExcurses: How does a slope answer \u0026ldquo;why\u0026rdquo;? # It doesn\u0026rsquo;t. Except when it does.\nA lot of ink has been spilled explaining the difference between correlation and causation. More generally, an \u0026ldquo;association\u0026rdquo; is not necessarily a \u0026ldquo;cause\u0026rdquo;. Yet, one key use of statistics, especially in science, is to help us decide if phenomenon X causes phenomenon Y. It seems unlikely that all of this effort is wasted.\nTo make a causal claim from statistics, more context is required about the data-generating process, such as that:\nValues of X were assigned randomly, yielding a randomized controlled trial (experiment). We can describe likely causes and interactions in a directed acyclic graph (DAG) or we can leverage other causal inference techniques. Previous discoveries or assumptions give us a theory or narrative connecting X and Y, and that has falsifiable predictions. Given this additional context and assuming that the model (here, a line) is not too far from the truth, the slope can be a meaningful explanation for why Y changes: because X did. Confidence intervals and their kin can tell us how certain we are about those relationships.\n\u0026ldquo;What?\u0026rdquo; = prediction of the future # Suppose we have our linear model above, and now we have a new situation described by new data, x=6.8x=6.8x=6.8. We want to predict yyy in this new situation. We can plug 6.8 in for xxx:\ny=2.6(6.8)+5.7=23.38 y = 2.6(6.8)+5.7 = 23.38 y=2.6(6.8)+5.7=23.38So far, so good. How sure are we that y=23.38y=23.38y=23.38? Can we use a confidence interval?\nNo. Just no. Don\u0026rsquo;t use a confidence interval for predictions. Especially those about the future.\nWhen making predictions, we need to account for two kinds of uncertainty:\nUncertainty about the parameters. Uncertainty about the error term. Uncertainty about the parameters is caused by (foolishly!) failing to collect an infinite amount of data. If you had an infinite amount of data,\nstandard error=standard deviation of the erroramount of data→0 \\text{standard error}=\\frac{\\text{standard deviation of the error}}{\\text{amount of data}} \\rightarrow 0 standard error=amount of datastandard deviation of the error​→0and your confidence interval would have zero width: you\u0026rsquo;d be certain about the mean effect of xxx on yyy.\nUncertainty about the error term is caused by (hopefully random) factors not included in the model. Even if you collect an infinite amount of data, there are still fluctuations in the observed values, deviations from the line fit to the (x,y)(x,y)(x,y) points. These deviations don\u0026rsquo;t get smaller just because we see more of them!\nPredictions have (1) some uncertainty that gets smaller when you have more data (uncertainty about parameters) and (2) some uncertainty that doesn\u0026rsquo;t go to zero (uncertainty in the error term). This means that prediction intervals are generally wider than confidence intervals. Put another way, using a confidence interval for predictions always overstates your certainty.\nThere are a variety of ways to calculate prediction intervals. My favorite is conformal prediction. A full tutorial is outside the scope of this post, but the idea is straightforward: resample the residuals.\nTrain your model on (X_train, y_train). Score the model on X_validation to get y_validation_prediction. Compute the residuals: residuals_validation = y_validation - y_validation_prediction. Sample from residuals_validation with replacement, say 1k times. Compute the desired quantiles, say (2.5%, 97.5%). One should be negative and the other positive. Make your predictions using X_future in the trained model. For each X_future, add the quantiles to get the desired intervals. This method works with any black-box model \u0026ndash; OLS, XGBoost, neural network, time-series \u0026ndash; any regression model. It is non-parametric and makes no assumptions (e.g., normality) about the distribution of the residuals. It is calibrated to give the desired coverage: as long as the data-generating process doesn\u0026rsquo;t change, it should be reliable.\nCare the roll the dice on your understanding? # Let\u0026rsquo;s nail down the difference between a confidence interval and a prediction interval with an example.\nSuppose we roll a 6-sided die 100 times.\nWhat is the expected value for one roll, i.e. the mean? For our experiment of 100 rolls, I got a sum of 338, or a mean of 3.38. How sure are we that that is the \u0026ldquo;true\u0026rdquo; mean for a fair die? The standard deviation of the observed rolls is 1.67, so the standard error of the mean is .167. A 95% confidence interval is [3.05, 3.71]. This does contain the \u0026ldquo;true\u0026rdquo; mean of 3.5. Excellent!\nNow, let\u0026rsquo;s predict the next roll. There\u0026rsquo;s a 95% chance that the next roll will be in [3.05,3.71][3.05, 3.71][3.05,3.71], right? Um, no. Exactly 0% will land in the confidence interval. The 95% conformal prediction interval is [1, 6]. Certainly more than 95% of rolls will fall in that range.\nThe derivation is left as an exercise for the reader.\n","date":"2025-07-03","externalUrl":null,"permalink":"/polimath/how-sure-are-you-confidence-v-prediction-intervals/","section":"Posts","summary":"Two common uses for statistical (or machine learning) models are:\nSomething happened. Why? Something will happen. What? Spoiler Alert # When reporting results, tell your audience how certain you are. This is essential context. “Why?”: When presenting parameters (drivers, slopes, causal effects, and the like) use something like a confidence interval, Bayesian credible interval, or bootstrapping the data. “What?”: When presenting predictions, use a conformal prediction interval, Bayesian posterior predictive interval, or the like. Forecasts must use these. “Why?” = inference about drivers # When we ask “why?” we are trying to make an inference about how the data was generated. Specifically, inferences about the parameters of the model, which can be thought of as measures of the relationships between features and the target.\n","title":"How sure are you? Confidence v. Prediction intervals","type":"post"},{"content":"","date":"2025-07-03","externalUrl":null,"permalink":"/polimath/tag/inference/","section":"Tags","summary":"","title":"Inference","type":"tags"},{"content":"","date":"2025-07-03","externalUrl":null,"permalink":"/polimath/tag/prediction/","section":"Tags","summary":"","title":"Prediction","type":"tags"},{"content":"","date":"2025-07-03","externalUrl":null,"permalink":"/polimath/tag/statistics/","section":"Tags","summary":"","title":"Statistics","type":"tags"},{"content":"","date":"2025-07-03","externalUrl":null,"permalink":"/polimath/tag/uncertainty/","section":"Tags","summary":"","title":"Uncertainty","type":"tags"},{"content":"I make birthday cakes for each of my boys. I don\u0026rsquo;t use fondant \u0026ndash; I strive for \u0026ldquo;tasty\u0026rdquo; \u0026ndash; so it can get a bit crazy.\nAnder turns 17 # To say Ander is into Mario Kart is an understatement. He is ranked among world record holders. This is serious. This year, he had been playing a lot with the character Yoshi, so he decided that\u0026rsquo;s who he wanted.\nThis is what he\u0026rsquo;s supposed to look like:\nI started by making one square and two round cakes, all nice and tall.\nThe outer shape came together better than I thought it would! Then, some frosting and the second layer: Yoshi\u0026rsquo;s nose.\nSome details finished him up, including mini-pro controllers and stars.\nHappy Mario Kart birthday!\n","date":"2025-06-23","externalUrl":null,"permalink":"/polimath/cake-ander-17-yoshi/","section":"Posts","summary":"I make birthday cakes for each of my boys. I don’t use fondant – I strive for “tasty” – so it can get a bit crazy.\nAnder turns 17 # To say Ander is into Mario Kart is an understatement. He is ranked among world record holders. This is serious. This year, he had been playing a lot with the character Yoshi, so he decided that’s who he wanted.\n","title":"Cake Ander-17: Yoshi!","type":"post"},{"content":"","date":"2025-06-22","externalUrl":null,"permalink":"/polimath/tag/measurement/","section":"Tags","summary":"","title":"Measurement","type":"tags"},{"content":"\u0026ldquo;Our net promoter score for the new version is 42.\u0026rdquo;\nOkay. So what? Is 42 good or bad? Should we continue rolling out the new version, or should we roll back to the previous version?\n\u0026ldquo;The previous version had an NPS of 35.\u0026rdquo;\nAh, some context! It looks like the new version is an improvement.\nWait, are we sure?\n\u0026ldquo;We surveyed 10 people.\u0026rdquo;\nHmm. That would be a no.\nWhen you report a number, provide a quantitative context.\nUsually this means:\nComparison: A second number (or more) that the audience is familiar with and to which they can compare the first number. Precision: Is there uncertainty about the numbers? How much? Significance: A sense of how much weight they should put on the difference between the two numbers when making a decision. Note that precision is not the same as significance.\nPrecision: Literally, think of this as the number of decimal places. Ways to describe this include statistical significance, standard errors, frequentist confidence intervals, Bayesian credible intervals, bootstrapped intervals, hypothesis tests, prediction intervals, or just a count of data points. Significance: Literally, is this difference important? This is not the same as statistical significance! This is a business determination, not a measurement. Let\u0026rsquo;s look at a simple example.\nIs 86% good or bad? Does this mean our retention efforts are working? We could add a comparison number explicitly by saying the industry average is Y% or last year we had 80%. A good choice is the percent difference with a familiar value, like last year\u0026rsquo;s retention.\nThis is better. Now we know that retention went up. Or did it? How precise are these numbers? One way to convey this would be to give a confidence or prediction interval, but these are not based on samples, so there is no sampling uncertainty. A sufficient way to convey precision is to say how many employees there are.\nIn this scenario, we retained 6/7 employees this year and 4/5 last year. Attrition was one person each year. There\u0026rsquo;s no real change, so let\u0026rsquo;s say so by stating the substantive significance.\nThis is good. We present the retention, a comparison, a sense of the precision, and the significance.\nNow, suppose we had the same retention rate and percent change, but a lot more employees.\nWe lost 1000 employees, but the long-term average tenure is 7 years per employee, just like the previous scenario, and it went up by two years (long-term average). We have a lot more data, so we are more precise: the 6% increase is not a quirk in the data. This is good news \u0026ndash; we should make sure the audience knows that.\nNote that the number we\u0026rsquo;re reporting (86%) is the same for the two scenarios, but the takeaway is very different because of the context.\n","date":"2025-06-22","externalUrl":null,"permalink":"/polimath/one-number-doesnt-do-it-context-matters/","section":"Posts","summary":"“Our net promoter score for the new version is 42.”\nOkay. So what? Is 42 good or bad? Should we continue rolling out the new version, or should we roll back to the previous version?\n","title":"One number doesn't do it: Context matters","type":"post"},{"content":"","date":"2025-06-06","externalUrl":null,"permalink":"/polimath/category/military/","section":"Categories","summary":"","title":"Military","type":"categories"},{"content":"The sixth graders stood in a rough line, like at their first dance, uncertain about approaching. Each held an envelope or sheet of paper. One, a boy I recognized as a friend of my son, came up to me. I put down my napkin and stood to face him.\n\u0026ldquo;Thank you for your service.\u0026rdquo;\nHe extended his hand. I shook it warmly. \u0026ldquo;You\u0026rsquo;re welcome.\u0026rdquo; He handed me his envelope and backed away.\nHe wasn\u0026rsquo;t sure how to show his appreciation, but I think he nailed it.\nAnother boy approached. \u0026ldquo;Thank you for your service.\u0026rdquo; Another envelope. Then three more boys. The girls, for some reason, chose other veterans at that breakfast, but every vet was thanked, every student completed their mission.\nHere\u0026rsquo;s what not to say when thanking a vet: \u0026ldquo;I know what you went through.\u0026rdquo; No, you don\u0026rsquo;t. Maybe I was injured. Maybe I killed people. Maybe I fought boredom while standing guard. Maybe I watched friends die.\nAlmost certainly I prepared myself mentally, physically, spiritually, to kill or be killed. That takes its toll. The culture I bunked with was toxic masculinity. Military thinking pervades my personality, even if it\u0026rsquo;s not clearly on display.\nThirty years later I\u0026rsquo;m still unpacking what that did to me, how it affects my relationships with the women in my life, with my kids, with the team I lead, with my boss. It shaped me socially in ways that make it hard for me to maintain male friendships. I lost some hearing. Too often I\u0026rsquo;m on guard.\nMy time in uniform shaped my views on discipline, honor, authority, accountability, respect, and discomfort. Shaped, not honed. Not all the shaping was for the better.\nI never saw action. My experience was nothing compared to those who have. But I carry scars, too, ones I feel every day.\nHere\u0026rsquo;s something you can ask, if you want: \u0026ldquo;May I ask you about your time in uniform?\u0026rdquo; They might say no. If they say yes, then ask about the positives. \u0026ldquo;What was something cool you saw?\u0026rdquo; \u0026ldquo;What was something you liked about it?\u0026rdquo; \u0026ldquo;What did you specialize in doing?\u0026rdquo; Then just listen. Let them talk, if they can.\nDon\u0026rsquo;t ask if they killed anyone. Just don\u0026rsquo;t. If the answer is no, you don\u0026rsquo;t get your voyeuristic thrill and you might embarrass them. If the answer is yes, their relationship with that fact is complicated, and not something discussed casually.\nI left the veterans breakfast, fighting back tears. I don\u0026rsquo;t know why I wanted to cry. Feeling raw, I put those envelopes aside for a couple of days before I read the notes within. Each was clumsy, but each boy seemed to understand a little of what I did and why I did it. My autistic son\u0026rsquo;s note thanked me for defending him being able to play Mario Kart. His understanding of my goals was incomplete, but not wrong.\n","date":"2025-06-06","externalUrl":null,"permalink":"/polimath/veterans-day-breakfast/","section":"Posts","summary":"The sixth graders stood in a rough line, like at their first dance, uncertain about approaching. Each held an envelope or sheet of paper. One, a boy I recognized as a friend of my son, came up to me. I put down my napkin and stood to face him.\n","title":"Veterans Day Breakfast","type":"post"},{"content":"Grandfather taught me that if you\u0026rsquo;re not sure what to say, try \u0026ldquo;Thank you.\u0026rdquo;\n\u0026ldquo;Here\u0026rsquo;s that report you asked for.\u0026rdquo;\n\u0026ldquo;Thank you.\u0026rdquo;\n\u0026ldquo;Have a good time!\u0026rdquo;\n\u0026ldquo;Thank you!\u0026rdquo;\n\u0026ldquo;I\u0026rsquo;d like to offer you some feedback.\u0026rdquo;\n\u0026ldquo;Thank you.\u0026rdquo;\n\u0026ldquo;F*** off!\u0026rdquo;\n\u0026ldquo;Thank you.\u0026rdquo;\nYou could say nothing. You could smile, frown, or stare confrontationally. You could say, \u0026ldquo;Okay\u0026rdquo;, \u0026ldquo;I will\u0026rdquo;, \u0026ldquo;On what?\u0026rdquo;, or \u0026ldquo;F*** you, too!\u0026rdquo;\nThose other approaches could minimize something nice or escalate something violent. Saying thank you is positive, genuine, and de-escalatory.\nWho can argue with being thanked?\nGrandfather was astute politically. As a leader in higher education, he continually faced the brutal* politics of academia. He was described as an \u0026ldquo;emergency medical technician\u0026rdquo; who would come in after university leadership had lost everyone\u0026rsquo;s trust, mop up the blood, lower faculty blood pressure, and set the next leader up for success.\nHe grew up in rural Louisiana and married a city girl from Baton Rouge. He worked the radio during WWII, then went north for his education in public policy. He lost his accent but always felt more comfortable in a sports coat, especially for dinner. He made a career out of building institutions (governmental and academic) that worked for everyone. One key to that was easing tensions and fomenting trust.\nHe taught me a lot. The value of persistence. The value of honesty and integrity. The importance of public service, of taking care of everyone. Poise. Curiosity and keeping up with technology. He read to me, and taught me to e-nun-ci-ate.\nHe taught me grace. And to be thankful.\n* Why are academic politics so brutal? Because the stakes are so low.\n","date":"2025-06-01","externalUrl":null,"permalink":"/polimath/thank-you-is-always-appropriate/","section":"Posts","summary":"Grandfather taught me that if you’re not sure what to say, try “Thank you.”\n“Here’s that report you asked for.”\n“Thank you.”\n“Have a good time!”\n“Thank you!”\n“I’d like to offer you some feedback.”\n“Thank you.”\n","title":"\"Thank you\" is always appropriate","type":"post"},{"content":"I wrote before about types of value generated by data science projects. What happens if you can\u0026rsquo;t report value in dollars?\nJames Q. WIlson classified bureaucratic agencies based on whether their inputs and outputs were measurable.\nWilson\u0026rsquo;s typology of agencies # Inputs measurable not measurable Outputs measurable \u0026ldquo;Production\u0026rdquo; — IRS; Social Security Administration \u0026ldquo;Craft\u0026rdquo; — wartime military; police not measurable \u0026ldquo;Procedural\u0026rdquo; — regulatory agencies; peacetime military \u0026ldquo;Coping\u0026rdquo; — State Department; teaching The IRS is a production agency because we can measure its inputs (number of returns processed, number of audits conducted) and outputs (revenue collected). SSA is the same: inputs checks issued and benefits paid, an output is lower senior poverty.\nThe wartime military is a craft agency because it is impossible, given the fog of war, to fully account for effort expended during battle, but you can tell whether the platoon took that hill. Police work is similar: it\u0026rsquo;s difficult to measure the effort of walking the beat, but we can measure the crime rate.\nThe peacetime military is interestingly the opposite, a procedural agency. We can document the specific training and exercises conducted, but how do you gauge the deterrent effect of a strong standing force when there\u0026rsquo;s no shooting? Regulatory agencies are similar: we can see the regulations created, but the deterrent effects are impossible to fully quantify. What are the monetary benefits of reduced air pollution?\nCoping agencies are particularly interesting because neither inputs nor outputs can be quantified. Consider a diplomat in the State Department. Is their effectiveness to be predicted by the length of their briefing paper? How much did that one paper decrease the likelihood of a trade war, and what would that trade war have cost the United States? With teaching, how do you measure the inputs (quality of instruction)? For outputs, we can use test scores, but those are affected more by household income and prior achievement than the current instruction. Plus, how do you measure intangible outputs like socialization?\nIf you acknowledge that the inputs of these agencies are costly and the outputs are valuable, then it\u0026rsquo;s reasonable to consider the relative value of these agencies. Should we invest in the Environmental Protection Agency (to help address climate change) or the State Department? Which costs more or addresses the bigger problem?\nEfficiency isn\u0026rsquo;t an answer # Efficiency (aka ROI) is defined as:\nefficiency=ROI=outputsinputs\\mathrm{efficiency} = \\mathrm{ROI} = \\frac{\\mathrm{outputs}}{\\mathrm{inputs}}efficiency=ROI=inputsoutputs​Suppose the inputs, outputs, or both aren\u0026rsquo;t measurable. Then you don\u0026rsquo;t have what you need to plug into this formula.\nExample: Staffing # You invest in a data engineer to partner with your data science team. You can measure their salary, but how do you quantify the increased satisfaction of your data scientists who can then focus on modeling instead of pipelines? This may decrease attrition among the data scientists. Does retention matter, and if so, how much?\nExample: CX Product # Your team builds a chatbot to let customers conduct business. Your KPI is a reduction in costs by reducing the number of agents required to assist customers. You\u0026rsquo;re surprised that the reduction in costs barely covers the cost of the team building the chatbot. You measured inputs and outputs, so the ROI must be well-defined, right?\nMissing inputs # To take on the chatbot project, you delayed data science work on GDPR compliance. This delay kept the risk of penalties for non-compliance high. You might estimate the opportunity cost like:\n(0.1% marginal probability of being sued)×($10M typical penalty for your industry)(\\text{0.1\\% marginal probability of being sued}) \\times (\\text{\\$10M typical penalty for your industry})(0.1% marginal probability of being sued)×($10M typical penalty for your industry)Are you sure about that 0.1%? How about that $10M? This estimate of opportunity cost could vary 100x given small changes in those guesses.\nMissing outputs # Your chatbot works! Then you hear from marketing about surveys that show:\nYour target demographic thinks tech-forward companies are more desirable. Prospective customers visiting your site see the chatbot and click to pages giving details. Current customers love it! It seems like this should increase acquisition and decrease churn, both valuable. How would you measure that value? You might be able to get a sense of #3, but a clean A/B test would be difficult. It\u0026rsquo;s hard to gauge #1 and #2, as marketers know, because you can\u0026rsquo;t prevent your control group from learning about your chatbot elsewhere. This means that you are certain to misvalue the chatbot.\nData science projects often have value that can\u0026rsquo;t be quantified. # ","date":"2025-06-01","externalUrl":null,"permalink":"/polimath/not-everything-valuable-is-measurable/","section":"Posts","summary":"I wrote before about types of value generated by data science projects. What happens if you can’t report value in dollars?\nJames Q. WIlson classified bureaucratic agencies based on whether their inputs and outputs were measurable.\n","title":"Not everything valuable is measurable","type":"post"},{"content":"","date":"2025-06-01","externalUrl":null,"permalink":"/polimath/category/politics/","section":"Categories","summary":"","title":"Politics","type":"categories"},{"content":"I make birthday cakes for each of my boys. I don\u0026rsquo;t use fondant \u0026ndash; I strive for \u0026ldquo;tasty\u0026rdquo; \u0026ndash; so it can get a bit crazy.\nArtemis turns 22 # Artemis has an imagination. As is typical, he asked for something unusual for his 22nd birthday cake. Here\u0026rsquo;s the reference art:\nHow hard could this be? What could possibly go wrong?\nI started out with a couple of basic cakes, side-by-side.\nThe box was set. So far, so good.\nThere were two challenges for this cake:\nThe rolled up lid. The intricate style of the reference box. The lid screamed, \u0026ldquo;cake roll!\u0026rdquo; I\u0026rsquo;d never done one, but as a fan of Ho Hos, I was ready to try. I found this\nrecipe and it was surprisingly straightforward.\nThen I worked on the rest of the design. I found this as reference art for the sardines:\nI combined that with Artemis\u0026rsquo;s drawn plan, and voila!\n","date":"2025-05-26","externalUrl":null,"permalink":"/polimath/cake-artemis-22-sardine-can/","section":"Posts","summary":"I make birthday cakes for each of my boys. I don’t use fondant – I strive for “tasty” – so it can get a bit crazy.\nArtemis turns 22 # Artemis has an imagination. As is typical, he asked for something unusual for his 22nd birthday cake. Here’s the reference art:\n","title":"Cake Artemis-22: Sardine Can","type":"post"},{"content":"Data science should deliver business value.\nWhen considering whether to invest in a data science project, itʼs useful to be clear about the kinds of value the project will bring.\nItʼs easy to fall into the trap of only valuing activity that can be measured in dollars. This misses a lot of real value, and can lead to the deprioritization of essential work that canʼt be easily connected to finances.\nNot everything valuable is measurable.\nI\u0026rsquo;ve identified six types of value that data science projects can generate:\nMoney Customer experience Employee experience Risk mitigation Legal compliance Strategic alignment 1. Money # You have a fraud detection model that analyzes online behavior to predict when a login attempt is likely fraudulent. If you improve the model\u0026rsquo;s sensitivity without increasing false positives, you can decrease losses due to fraud without making things more difficult for non-fraudulent customers. This is measurable in dollars.\n2. Customer experience # You build a customer-facing chatbot for your site that answers questions and can even perform some actions. This might increase conversion rate or average transaction value, but gauging how much is attributable to the chatbot is difficult and will undervalue the work. A way to gauge the non-monetary value of the chatbot would be to ask customers for feedback (Net Promoter Score, thumbs up/down). Positive customer experience is itself valuable, even if it canʼt be converted neatly into dollars and cents\n3. Employee experience # You create a search engine that helps employees find particular datasets. You could try to measure the value in terms of (time saved) x (average wage), but this misses a lot of the value. How would you quantify money saved through better retention because the search engine makes analysts less frustrated? The chain of reasoning from measurable action to measurable financial impact is long, and the accuracy of each link is low, so financial estimates could vary wildly. But if it does improve employee experience, that itself is valuable\n4. Risk mitigation # Your Internal Audit team is tasked with ensuring the chain of custody for your products brought back in for repair. The process includes taking photos of products on the way in and on the way out. You developed computer vision models to gauge the quality of those images. Are they in focus? Are they zoomed in? If you detect low-quality images, Internal Audit can arrange for training or better equipment to improve the process, increasing the reliability of the chain of custody, reducing risk.\n5. Legal compliance # Compliance with privacy laws like the California Consumer Privacy Act and the EUʼs GDPR comes at a cost, but the financial benefits are difficult to calculate. If the average lawsuit against companies in your industry arising from practices that donʼt comply is $10M, do we say compliance reduces our risk by half? three-quarters? Precision is out the window. But valuing compliance is meaningful on its own.\n6. Strategic alignment # Your Chief Strategy Officer has intel that it\u0026rsquo;s time to strike in a new geographic area. You build a model to recommend sites for company locations in the new area. Each location can generate financial value, but the presence in that area may have larger strategic value.\nHaving multiple ways to gauge the business value of a project means that you can\u0026rsquo;t prioritize with:\nprojects_df.sort_values(by='business_value', ascending=False)\nSo what use is this?\nWhen discussing where to invest data science resources, frame the discussion so that the real value of the work can be fairly considered.\n","date":"2025-05-26","externalUrl":null,"permalink":"/polimath/the-business-value-of-a-data-science-project/","section":"Posts","summary":"Data science should deliver business value.\nWhen considering whether to invest in a data science project, itʼs useful to be clear about the kinds of value the project will bring.\n","title":"The business value of a data science project","type":"post"},{"content":" Stephen Haptonstahl, Ph.D. # AI / Data Science team builder and leader\nVeteran Mathematician Social scientist Teacher Advocate for autistics Baker of moderately interesting cakes Management is allocating resources. Leadership is influencing people toward goals.\nAs learned during my time in uniform To get in touch, find me on LinkedIn.\n","externalUrl":null,"permalink":"/polimath/about/","section":"Polimath","summary":"Stephen Haptonstahl, Ph.D. # AI / Data Science team builder and leader\nVeteran Mathematician Social scientist Teacher Advocate for autistics Baker of moderately interesting cakes Management is allocating resources. Leadership is influencing people toward goals.\n","title":"About Stephen","type":"page"}]