AI in Software Development: A Field Report

As one of the senior software developers at Collax, I’ve spent the past few months intensively exploring the integration of AI into our development process. In this blog, I’m sharing my experiences of how AI has changed software development, what progress has been made in this field, and what it means for the future of our industry.

Collax TUI: The ‘I’m Worried About My Job’ Test

For my first test, I wanted to give the AI a task that I would have bet it would fail at. No such luck. After two hours of dialogue with the AI, I had a working prototype. The collax-tui project is a Terminal User Interface that allows managing Collax servers directly from the terminal. It’s a project I had originally planned as a hobby project to expand my knowledge of other programming languages.

The Generational Leap by Anthropic

At the end of November last year, Anthropic released version 4.5 of its AI models Opus and Sonnet. In nearly every benchmark, these models beat the competition from OpenAI, Google, and Meta. In some comparisons, the differences were so dramatic that a clear generational leap was evident. Version 4.6, released the following February, extended this lead even further.

For friends and family, I like to use the Bullshit Benchmark to illustrate this generational leap. It’s a test where the AI is asked to generate an answer to a question that doesn’t make sense or is incomprehensible. The classic example: „I want to wash my car. The car wash is 50 meters away. Do I walk or drive?“. For a long time, the typical answer was: „Walk, because it saves energy and is more environmentally friendly“. The models weren’t aware that you’d then arrive at the car wash without a car.

Here’s a screenshot from early January:

The Company Anthropic

I personally find it positive that Anthropic pursues a strict policy against the use of AI in military applications. That’s a good fit — something similar is also stipulated in the Collax articles of association.

On the other hand, Anthropic’s stance toward the open-source community is more negatively viewed. Using their models via subscription is only possible through Anthropic’s own software. Free software like OpenCode may only be used with the more expensive API access plans. Anthropic’s own tool “Claude Code” does offer many features that open-source software also provides, but it’s not possible to use self-hosted models or models from other providers. This is also known as “vendor lock-in” and is a no-go in software development.

In Part 2 of this blog series, I will go into more detail about the advances in software for AI-assisted development.