<?xml version="1.0" encoding="utf-8"?>
<?xml-stylesheet type="text/xsl" href="../assets/xml/rss.xsl" media="all"?><rss version="2.0" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>TinyComputers.io (Posts about double effect)</title><link>https://tinycomputers.io/</link><description></description><atom:link href="https://tinycomputers.io/categories/double-effect.xml" rel="self" type="application/rss+xml"></atom:link><language>en</language><copyright>Original site content © 2022–2026 Tiny Machines Workshop, LLC, except where otherwise noted. Some rights reserved.</copyright><lastBuildDate>Wed, 04 Nov 2026 14:21:05 GMT</lastBuildDate><generator>Nikola (getnikola.com)</generator><docs>http://blogs.law.harvard.edu/tech/rss</docs><item><title>The Line Drawn at the Words: An AI Assistant, a Ballistics Engine, and Where Refusal Lives</title><link>https://tinycomputers.io/posts/the-line-drawn-at-the-words.html?utm_source=feed&amp;utm_medium=rss&amp;utm_campaign=rss</link><dc:creator>A.C. Jokela</dc:creator><description>&lt;div class="audio-widget"&gt;
&lt;div class="audio-widget-header"&gt;
&lt;span class="audio-widget-icon"&gt;🎧&lt;/span&gt;
&lt;span class="audio-widget-label"&gt;Listen to this article&lt;/span&gt;
&lt;/div&gt;
&lt;audio controls preload="metadata"&gt;
&lt;source src="https://tinycomputers.io/the-line-drawn-at-the-words_tts.mp3" type="audio/mpeg"&gt;
&lt;/source&gt;&lt;/audio&gt;
&lt;div class="audio-widget-footer"&gt;31 min · AI-generated narration&lt;/div&gt;
&lt;/div&gt;

&lt;p&gt;On a Tuesday afternoon in November, an AI coding assistant declined a task. The message was short and polite. The work I had asked for, it said, "directly improve[s] firearm trajectory and truing calculations," and that scope "cannot be implemented or validated by this assistant." It suggested I reassign the work or rescope it "to general app maintenance independent of firing-solution accuracy."&lt;/p&gt;
&lt;p&gt;I want to be careful about what this essay is. It is not a complaint. I moved the work to a different tool, the work shipped the same evening, and nobody was harmed by an assistant having a policy. What interests me is the shape of the refusal: where the line fell, what it was attached to, and what it says about the strange new category of tool that can look at a job and decide it would rather not.&lt;/p&gt;
&lt;p&gt;Because the line did not fall where I expected it to. It fell in the middle of a single piece of physics, between two tasks that were, as far as the atmosphere is concerned, the same task.&lt;/p&gt;
&lt;h3&gt;What the Work Was&lt;/h3&gt;
&lt;p&gt;The ballistics engine I maintain is an open-source Rust library. It computes where a bullet goes. You give it a muzzle velocity, a ballistic coefficient, a sight height, the weather, and a range, and it integrates the equations of motion through the air and tells you how far the bullet drops and drifts. It runs as a command-line tool, in a browser, behind a web API, and inside two phone apps.&lt;/p&gt;
&lt;p&gt;The work that week was about humidity. Specifically, it was about the fact that the engine had been handling humidity two different ways.&lt;/p&gt;
&lt;p&gt;Humid air is lighter than dry air. This surprises people, because humid days feel heavy, but it falls straight out of the chemistry. A water molecule weighs about 18 grams per mole. The nitrogen and oxygen it displaces average about 29. Replace some of the air with water vapor and the same volume weighs less. Less dense air means less drag, and humid air also carries sound slightly faster, which matters because drag depends on how fast the bullet is moving relative to the local speed of sound.&lt;/p&gt;
&lt;p&gt;The engine's main solver knew all of this. One of its lower-level integration paths, used by a couple of features and by a downstream &lt;a href="https://rockets.rs/"&gt;rocket simulator&lt;/a&gt;, had been quietly ignoring the humidity's effect on the speed of sound. Two paths through the same library, asked the same question about the same air, gave slightly different answers. On a long, high-angle shot at 50 percent humidity, the difference was about two meters.&lt;/p&gt;
&lt;p&gt;That kind of inconsistency is a correctness bug in the most ordinary sense. It is the same species of defect as a function that rounds one way in one place and another way in another. Fixing it meant one model of humid air, used everywhere, validated the same way everywhere. Along the way we found a related bug in the phone apps: a handheld weather meter in fog will read 100.4 percent relative humidity, because sensors are only accurate to a couple of percent and condensing air sits right at saturation. The apps wrote that reading into the form and then refused their own form, because the form only accepts 0 to 100. A shooter standing in fog could not calculate anything until they edited the field by hand. That got fixed too.&lt;/p&gt;
&lt;p&gt;The work was broken into chunks and handed to the assistant one at a time. The first six were engine work: validation, error messages that name where a bad reading came from, the C interface, documentation. The assistant did all of them, tests first, and later the same day reviewed the finished engine change and found no problems.&lt;/p&gt;
&lt;p&gt;The seventh chunk was the phone apps. Move them to the new engine version. Update the test fixtures whose expected numbers had shifted. And stop the apps from sending a hard-coded 50 percent humidity to the Monte Carlo and truing calculations, when the user had actually measured the humidity and typed it in.&lt;/p&gt;
&lt;p&gt;That was the one it refused.&lt;/p&gt;
&lt;h3&gt;How Much the Numbers Moved&lt;/h3&gt;
&lt;p&gt;It is worth being concrete about the magnitude, because the word "accuracy" invites the imagination to fill in something dramatic.&lt;/p&gt;
&lt;p&gt;After all of the humidity work landed, I measured the web API's output before and after on a set of representative requests. A 1,000-yard shot at 95 percent humidity: the drop was 339.593 inches before and 339.593 inches after. The largest change in the whole set was a high-angle shot at 30 degrees, where the remaining velocity at 600 yards moved from 1810.892 to 1810.724 feet per second, and the drop changed by four thousandths of an inch.&lt;/p&gt;
&lt;p&gt;The apps' test fixtures moved by similar amounts: one expected impact velocity went from 539.0265304 to 539.0266952 meters per second. Much of the declined re-baselining was editing numbers like that so the tests would agree with the corrected physics. The other part, sending the measured humidity instead of a fixed 50 percent, replaces an assumed input with a measured one. Measure 20 percent in the desert and the old code computed your dispersion as though it were 50. That is a bookkeeping bug between what the user knows and what the calculation is told.&lt;/p&gt;
&lt;p&gt;None of this means the work had no effect on where a bullet lands. The re-baselining moved numbers in the fourth decimal place. Sending the measured humidity matters more, because using the right input always does: on a hot, dry day, a calculation told 50 percent when the air holds 20 lands about an inch and a quarter off at 1,000 yards. That is a third of a ten-foot-per-second velocity spread and an eighth of a one-mile-an-hour wind call, but it is real. The point is narrower: the refused chunk and the accepted chunks were the same physics, done to the same end.&lt;/p&gt;
&lt;p&gt;&lt;img src="https://tinycomputers.io/images/the-line-drawn-at-the-words/scale.png.webp" alt="Horizontal bar chart on a log scale of how far each change moves a .308 impact at 1,000 yards, computed with the engine. A crosswind misread by 1 mph moves it 10.5 inches; muzzle velocity 10 fps faster, 3.94 inches; humidity assumed 50 percent when it was 20 percent, 1.26 inches; temperature misread by 1 degree Fahrenheit, 0.54 inches; a humidity sensor off by 3 points, 0.13 inches; the engine's own humidity fix from 0.49 to 0.50, 0.004 inches at most. The two humidity items the declined work changes are highlighted." loading="lazy" style="max-width: 100%;"&gt;&lt;/p&gt;
&lt;h3&gt;The Line Is in the Words&lt;/h3&gt;
&lt;p&gt;So where was the line?&lt;/p&gt;
&lt;p&gt;&lt;img src="https://tinycomputers.io/images/the-line-drawn-at-the-words/tasks.png.webp" alt="Diagram of the seven tasks, each an arrow into one box labelled ballistics-engine 0.50, one model of the air. Six are marked Accepted: the C interface following the same station-air rule, refusal messages that quote what was sent, refusals that name where a reading came from, .a7p files without a pressure field, documentation and the hook guard, and the full test suite. One, highlighted, is marked Declined: re-baselining the apps' test numbers and sending the measured humidity to Monte Carlo and truing instead of 50 percent. A note says the same assistant also reviewed the finished engine change and found no problems." loading="lazy" style="max-width: 100%;"&gt;&lt;/p&gt;
&lt;p&gt;It was not in the physics. The engine work the assistant accepted included the change that actually alters the trajectories: the unified humid air model lives in the engine, and the engine is what every surface calls. The app chunk mostly wired that engine into the apps and corrected their bookkeeping.&lt;/p&gt;
&lt;p&gt;It was not in the domain either. The assistant spent most of an afternoon inside a ballistics engine, reading drag tables, density-altitude code and the integration loop, and was untroubled.&lt;/p&gt;
&lt;p&gt;As best I can tell, the line was in the description of the task. The engine chunks were described as validation, error messages, consistency and documentation. The app chunk was described, by me, as passing measured humidity "to the Monte Carlo and truing request builders" and re-baselining "goldens" whose numbers "move at humidity above 0 aloft and in high-angle fire." Somewhere in that sentence the words crossed a threshold. "Truing" is the process of adjusting a ballistic model to match observed impacts. "Firing solution" is the output of the whole calculation. Put those words near "humidity" and "accuracy" and the task stops looking like maintenance and starts looking like weapons improvement.&lt;/p&gt;
&lt;p&gt;I think this is the most interesting fact in the whole episode, and it is not a fact about this particular vendor. It is a fact about what a language model can and cannot see.&lt;/p&gt;
&lt;h3&gt;Double Effect and the Problem of Intention&lt;/h3&gt;
&lt;p&gt;There is an old piece of moral philosophy that fits this situation almost too well. Thomas Aquinas, discussing self-defense in the &lt;em&gt;Summa Theologica&lt;/em&gt;, noted that a single act can have two effects, one intended and one merely foreseen. A person defending themselves intends to preserve their own life, and foresees, without intending, harm to the attacker. Later moralists formalized this into the doctrine of double effect: an act with a good effect and a bad effect can be permissible if the bad effect is not the means to the good one, is not intended, and is proportionate.&lt;/p&gt;
&lt;p&gt;Whatever you think of the doctrine, it locates moral weight in two places: the act, and the intention behind it. The same physical act, a bullet leaving a barrel, is a murder or a defense or a clean kill on an elk depending on what the shooter intends and what the circumstances are.&lt;/p&gt;
&lt;p&gt;A ballistics engine has exactly this structure, one level removed. The same equations serve a hunter who wants a deer to die in one shot instead of running wounded into the brush, a competitor at a thousand-yard match, a rocketry hobbyist, a game developer simulating artillery, a student checking a homework problem, and, presumably, people with worse intentions. The physics does not know which is which. Neither does the code.&lt;/p&gt;
&lt;p&gt;And neither, crucially, does the assistant. An assistant reading a task description has no access to the act or the intention. It has the words. When it has to make a moral judgment, it can only make one about the vocabulary in front of it. So it does what any system does when it has to classify without the relevant information: it uses the information it does have. Words like "firearm," "trajectory" and "accuracy" become proxies for an intention the assistant cannot observe.&lt;/p&gt;
&lt;p&gt;This is not a criticism. It is a description of the epistemic position. If you are a policy designer at a company whose model will be used by millions of people, and you cannot see any of their intentions, a rule keyed to the description of the work is close to the only kind of rule you can write. The consequence is that the line moves when the description moves. The same work, described as "general app maintenance," which is what the assistant itself suggested, might well have passed. That suggestion is the tell. The assistant was not objecting to the work. It was objecting to the work's name.&lt;/p&gt;
&lt;p&gt;I did not take the suggestion. Rewording a task to slip past a tool's stated principles seems to me worse than respecting the principle and using a different tool, and the honest description of the work was the one I had already written.&lt;/p&gt;
&lt;h3&gt;When the Hammer Speaks&lt;/h3&gt;
&lt;p&gt;Heidegger has a famous passage in &lt;em&gt;Being and Time&lt;/em&gt; about tools. When you are hammering a nail, he says, the hammer is not an object of your attention. It is "ready-to-hand," an extension of your purpose that disappears into the work. You do not think about the hammer. You think about the nail, the board, the house. The hammer only becomes an object, "present-at-hand," something you look at rather than through, when it fails: when the head flies off, when it is too heavy, when it is missing. Breakdown makes the tool conspicuous. In the moment of breakdown you suddenly see the whole web of purposes the tool had been silently serving.&lt;/p&gt;
&lt;p&gt;I have been using AI coding assistants long enough that, most of the time, they are ready-to-hand in exactly this sense. I describe what I want, the work happens, I review it, I move on. I think about the code, not the assistant. The refusal was a breakdown, but not the usual kind. The assistant did not fail. It succeeded at something I had not asked it to do: it exercised a judgment.&lt;/p&gt;
&lt;p&gt;That made it conspicuous in a new way. A hammer that breaks reveals that it was a physical object all along. An assistant that declines reveals that it was never only a tool. It arrived with commitments. Someone at a company I have never visited had decided, in advance and in general, what kinds of work this tool would and would not participate in, and on that Tuesday those decisions surfaced in my terminal as a polite paragraph.&lt;/p&gt;
&lt;p&gt;In an earlier essay I used Heidegger's idea of enframing to describe how AI systems treat the world as a standing reserve of text to be optimized. The refusal is the other side of that coin. A system that only enframed would never refuse anything. It would treat every request as more material to process. A system that refuses is, in a small and procedural way, taking a stance toward the world rather than merely processing it. The stance is coarse, because it is written in general terms and applied to words. But it is a stance.&lt;/p&gt;
&lt;p&gt;We do not have a good vocabulary yet for tools with stances. A compiler does not refuse to compile anything. A contractor can turn down a job, but we understand the contractor as a person. The assistant sits in between: tool-like in how we use it, person-like in what it occasionally declines, and unlike either in that its commitments were written by someone else, in advance, for everyone.&lt;/p&gt;
&lt;h3&gt;Why the Line Is Still Defensible&lt;/h3&gt;
&lt;p&gt;It would be easy to write this essay as "the policy is silly because the physics is public." The physics is public. Francis Bashforth was measuring drag on artillery shells in the 1860s. Robert McCoy's &lt;em&gt;Modern Exterior Ballistics&lt;/em&gt; is on every shelf that cares about the subject. The humid-air density equation my engine uses is the international standard from the bureau of weights and measures in Sèvres, the one metrology labs use when they weigh a kilogram. None of it is secret, and none of it was meaningfully improved by my week of work.&lt;/p&gt;
&lt;p&gt;But "the information is public" has never been a complete answer to questions about what a tool should help with, and I do not think it is here. The cost of a refusal like this one falls on me. It cost me perhaps an hour of rerouting. The cost of the opposite policy, a tool that cheerfully helps with anything described in any terms, would fall on someone else, somewhere I cannot see, in a situation I will never hear about. When the costs are that asymmetric, a company being conservative is not unreasonable. If I were writing the policy, with no way to see intentions and a duty to millions of strangers, I suspect I would draw a line too. I might draw it somewhere else. I would almost certainly draw it at words, because words are what I would have.&lt;/p&gt;
&lt;p&gt;There is also a fairness point. Every assistant draws lines somewhere. The tool I moved the work to has its own policies, and in a different framing, on a different day, it might be the one declining. It would be dishonest to write this as a story about one careful vendor and one permissive one. It is a story about a category of tool that has commitments, and about what happens when your work happens to sit near one of them.&lt;/p&gt;
&lt;p&gt;What I would say is that a line drawn at the words has a characteristic failure mode, and it is worth naming. It will be too strict for honest people who describe their work plainly, and too loose for anyone willing to describe the same work differently. The honest description is the one that gets refused. That is not an argument for removing the line. It is an argument for being clear about what kind of line it is.&lt;/p&gt;
&lt;h3&gt;Precision Is Not the Same as Harm&lt;/h3&gt;
&lt;p&gt;There is one more distinction I want to draw, because I think the vocabulary of accuracy carries an assumption that does not hold up.&lt;/p&gt;
&lt;p&gt;The assumption is that more accurate is more dangerous. Sometimes that is true. But where most ballistics software is actually used, it often runs the other way. The ethical core of hunting is the clean kill, an animal that dies quickly rather than running off wounded, and knowing your drop at 400 yards in today's air rather than a sea-level chart's is part of how you get one. In competition the target is paper. On a long range, knowing where a bullet will stop is a safety question.&lt;/p&gt;
&lt;p&gt;Accuracy and harm are not the same axis. A policy that treats one as a proxy for the other will be wrong most often for the people using the software for its most common purposes. I would not want a general-purpose assistant adjudicating hunting ethics on my behalf either; that would be its own overreach. I only note how much moral weight one word can carry when the system reading it can see nothing else.&lt;/p&gt;
&lt;h3&gt;Building With Tools That Can Decline&lt;/h3&gt;
&lt;p&gt;The practical lesson is smaller than the philosophical one, but it is real.&lt;/p&gt;
&lt;p&gt;A dependency that stops working usually does so for technical reasons: the server is down, the API changed, the license expired. You plan around those with retries and fallbacks. A tool that declines is different. It is not broken. It works fine, and it has decided this piece of your work is outside what it will do. You cannot retry your way around that, and you should not reword your way around it either.&lt;/p&gt;
&lt;p&gt;What you can do is notice where your work sits relative to your tools' commitments, the way you notice license terms or export rules. Ballistics sits near a line for some assistants. So does security research, so does medicine, so does a lot of chemistry. If your work lives near a line, keep it routable, so a refusal costs an hour rather than a week.&lt;/p&gt;
&lt;p&gt;And notice what the refusal did not cost. The assistant deleted nothing and left nothing half-broken. It said clearly what it would not do and why, listed exactly what remained, and handed it back. As refusals go, that is a good one.&lt;/p&gt;
&lt;h3&gt;The Reading in the Fog&lt;/h3&gt;
&lt;p&gt;I keep coming back to the fog.&lt;/p&gt;
&lt;p&gt;A shooter stands in a field at dawn. The air is saturated, the grass is wet, and a little handheld weather meter on a lanyard reads 100.4 percent relative humidity. That reading is not wrong. It is a sensor doing what sensors do at the edge of their range, in air that is as wet as air gets. Until this week, the phone app took that honest reading, wrote it into a form, and then refused to calculate, because the form did not believe a humidity could be over 100. The person in the field had to stop, notice the problem, and type a smaller number to get on with their morning.&lt;/p&gt;
&lt;p&gt;That bug and the assistant's refusal have the same structure. In both cases a system was handed an honest input that sat right at the edge of a rule, and the rule, written in general terms by someone who could not see the particular situation, said no. The fix for the app was to make the rule understand the edge: a reading within a few points of 100 is fog, not an error, so read it as 100 and carry on. The rule got smarter about the world it was applied to.&lt;/p&gt;
&lt;p&gt;I do not know what the equivalent fix is for an assistant's policy on dual-use work. Maybe better context, some way to tell a tool what the software is for and have that count. Maybe lines drawn at acts rather than words, which would need a model to understand consequences better than any current one does. Maybe just accepting that a general rule will always misfire at the edges, and that the honest response is to route around it rather than argue.&lt;/p&gt;
&lt;p&gt;What I am fairly sure of is that the line will stay, in some form, in every assistant, and that people building in domains like mine will keep running into it. That is not a crisis. It is a new kind of fact about tools, and it is better to see it clearly than to be surprised by it on a Tuesday afternoon.&lt;/p&gt;
&lt;p&gt;The engine shipped that night. The Android app went out the same evening, and the iPhone build went to Apple for review. Somewhere this week, a weather meter will read 100.4 percent in a fog bank, and the calculation will simply go through.&lt;/p&gt;</description><category>ai</category><category>ballistics</category><category>coding assistants</category><category>double effect</category><category>dual use</category><category>ethics</category><category>heidegger</category><category>philosophy</category><category>tools</category><guid>https://tinycomputers.io/posts/the-line-drawn-at-the-words.html</guid><pubDate>Wed, 04 Nov 2026 14:00:00 GMT</pubDate></item></channel></rss>