Schallbert's Blog

Hardware Security Key: add SSH

Schallbert — Sun, 03 May 2026

Motivation

In my post about hardware security keys, I wrote that I want to secure as many of my connections as possible without passwords provided by my new HSK. I use SSH connections for various services and purposes: I access my VPS, use services such as GitHub, Gitea and my own Gitea instance and run backups on the local network via scp, which uses the SSH protocol.

Setting up SSH access for hardware security keys

HSKs such as my Yubikey can store private keys for SecureShell access. This means you no longer need to store them locally on every device (although this remains configurable; see below). Furthermore, they are better protected against unauthorised access than if stored on a computer.

So I’m trying to switch my GitHub SSH access to HSK and am following Yubico’s guide to passwordless authentication for GitHub. The procedure for generating FIDO-compliant SSH keys is the same across different key manufacturers. I’m also linking to the Nitrokey guide for reference.

What doesn’t work

However, even with Yubico’s own recommendations, I am unable to achieve the desired result on my operating system and with the standard installation of openssh. The command assembled from the guide is:

schallbert@machine:~# ssh-keygen -t ed25519-sk -O resident -O verify-required -O application=ssh:github -O user=schallbert -C 'schallbert@github.com'

Once generated, we must store the public key (file extension .pub) in the web application of the respective service under the SSH Access menu item.

Meaning of the command and the most important options

This command¹ tells ssh-keygen to generate a key of type -t ed25519-sk for me. The suffix -sk indicates that the key should be stored on a hardware security key - in my case, the Yubikey.

To understand the -O resident flag, we need to know that the FIDO authenticator key, generated as with -sk, consists of two parts: a key handle part, which remains on the computer’s hard drive, and the private key itself, which exists only on the HSM. Only together can they correctly respond to authentication requests. The flag itself, however, instructs the program to write both parts of the key to the Yubikey. This makes it easier to use multiple devices with the same key: the key handle part can be read from the Yubikey and stored on other end devices. On the other hand, however, simply losing the Yubikey is now enough to grant potential attackers SSH access.

-f filename specifies the names for the key files. This is particularly useful when you need to manage multiple SSH connections or HSMs. I’ve created a naming convention to avoid confusion. It’s based on the syntax used to read keys from an HSM (ssh-keygen -K).

-f <keyType>_<protocol>:<service>_<user>_<device>

As a concrete example, my filename here would be as follows:

-f ed25519-sk_ssh:github_schallbert_yubikey-main

⚠️ Note when using Windows: The colon : is not allowed as a character in filenames. It is replaced by an underscore _, which must also be reflected in the config file (Ctrl+H -> replace : with _)

-C schallbert@github.com adds a comment to the key file itself, such as a contact address. This gives the key a label, which makes it easier to manage when dealing with multiple keys.

-O application=ssh:github is simply for neat listing in the Authenticator and ensures that I cannot confuse one key with another. Otherwise, it would simply say ssh:keygen and I would no longer be able to see which services I have stored keys for. Also helpful to discriminate keys when using the -resident option in congunction with key extraction ssh-keygen -K.

-O user=schallbert has no practical use for me, as SSH is not interested in this flag and, contrary to my expectations, the Yubikey Authenticator does not display the username.

-O no-touch-required prevents the hardware security key from having to be touched every time the key is used. Useful for very frequent use, though it compromises security due to the possibility of automation.

-O verify-required specifies that the HSM’s PIN must be entered before the key can be used. Although intended as an additional security measure, this causes problems for me in my shell. Specifically, the PIN is requested once, but the ssh-agent does not generate a prompt in my console every time. As soon as I try to connect via SSH in another session, I receive an error message: agent refused operation:

schallbert@machine:~# ssh -T git@github.com
sign_and_send_pubkey: signing failed for ED25519-SK "schallbert@github.com" from agent: agent refused operation
root@opnsense: Permission denied (publickey).

What was the problem?

After a long, frustrating troubleshooting session (What am I doing wrong? Does it work with the secondary key? Why does it work the first time but not afterwards? What if I regenerate the keys? What happens on other computers? What does ‘AI’ say about this? - The latter certainly took an hour and left me completely stuck), I try a different command. That works straight away.

What does work, however

schallbert@machine:~# ssh-keygen -t ed25519-sk -O application=ssh:github -C 'schallbert@github.com' -f ed25519-sk_ssh:github_schallbert_yubikey-main

If I omit verify-required, I can log in successfully and the SSH agent runs fine.

What `verify-required` requires

This post sheds some light on my issue. The terminal session I use to create the SSH key pair temporarily stores the PIN entry enforced by the verify-required flag. That is why the login works initially. If I log in later or in a different window, as I did above to check git pull in one of my repos, for example, the SSH agent does not ask for my PIN.

This is likely because the SSH agent itself cannot display prompts to the requesting programme. A submodule of OpenSSH provides a solution: openssh-askpass . This can be passed to the agent, which then transmits the PIN prompt to the user via askpass.

Installing ssh-askpass

My operating system did not have ssh-askpass installed by default, so I’ll install it here. It uses the package manager DNF (‘Dandified YUM’) instead of apt or other common managers. In my opinion, the naming convention isn’t 100% consistent: whilst I address the ssh programme in the terminal as ssh-agent, I have to type openssh-agent for the installation.

# verify-required needs openssh subpackage through Fedora's package manager
schallbert@machine:~# sudo dnf install openssh-askpass
Installing:
 openssh-askpass
[...]
Complete!

Now I want to tell the agent that it can use ssh-askpass to request the PIN. To do this, I need to know the path to the submodule.

schallbert@machine:~# which openssh-askpass
/usr/bin/which: no openssh-askpass [...]

Strange, I installed it above! Let’s have a look what the package manager has to say.

schallbert@machine:~# dnf repoquery --list openssh-askpass
[...]
/usr/libexec/openssh/ssh-askpass

Ah, openssh hides in libexec. Well, then I can link ssh-agent to ssh-askpass.

schallbert@machine:~# eval "$(ssh-agent -s; SSH_ASKPASS=/usr/libexec/openssh/ssh-askpass)"
Agent pid 16747

I check the connection to GitHub and, as expected, a dialogue box appears asking for my PIN. I then need to tap the HSK, after which I receive a success message. To ensure that this all works in a new terminal and after restarting the computer, I add a reference to the key in my configuration file.

# ~/.ssh/config
Host github
 User git
 Hostname github.com
 PreferredAuthentications publickey
 IdentityFile ~/.ssh/ed25519-sk_ssh:github_schallbert_yubikey-main

To be on the safe side, restart the agent and create new terminal sessions. If any login details were cached, this will reset them. When done, validate SSH connectivity.

 schallbert@machine:~# eval "$(ssh-agent -k)" && eval "$(ssh-agent -s)"

Annoying: Gnome pop-up ‘Allow inhibiting shortcuts’

The only downside is that I now get an extra pop-up when connecting via SSH. This is triggered by the gnome graphical user interface. Unfortunately, I can’t find an option to save the relevant setting permanently. As a result, it appears every time I connect with verify-required. With Gnome, the problem is already known to affect other applications, but apparently has not yet been resolved.

Transferring to Windows systems

I generated my keys using the -O resident option. It makes sense to test the extraction of the key-handle on a completely different system: Windows.

I open the command line and type

schallbert@windows-machine:~# ssh-keygen -K
Enter PIN for authenticator:
You may need to touch your authenticator to authorize key download.
Provider "internal" returned failure -1
Unable to load resident keys: invalid format

Invalid format? It’s often mentioned online that the keys sometimes need a line feed character at the end to be recognized correctly. However, I cannot modify them without downloading them from HSK. So I manually transfer the public key and key handle to the computer and try out the git pull command as a test. And look: it works!

So if you want to use the keys with different operating system families, the -resident option doesn’t seem to be of any use. The command ssh-keygen -K does not allow granularity: It will load all resident keys from the HSK at once.

VPS: Be careful when replacing keys

Another tip from personal experience: If you’ve already stored SSH keys with a service provider that hosts live-resources itself (in my case, a server as a VPS), new SSH keys are usually not passed on to that resource via the web frontend.

Consequently, the “old” keys are still needed, and the new ones aren’t used, even though they’re stored in the web app!

The new keys must be manually added to the destination.

I follow the instructions from my host Hetzner using the ssh-copy-id command.

Manually store new keys on the VPS

schallbert@machine:~# ssh-copy-id -i ./ed25519-sk_ssh:server_schallbert_yubikey-main.pub server

Here, server is the SSH alias for my VPS. The connection is established using the familiar “old” key, and the “new” one is saved. But something isn’t working as expected:

/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: WARNING: All keys were skipped because they already exist on the remote system.
		(if you think this is a mistake, you may want to use -f option)

Was the key transferred successfully?

So I log back into my server and check whether the new key actually appears in the ~/.ssh/authorized_keys file. It doesn’t. So I run the command above again, this time with the -f (Force) option enabled. And sure enough, the transfer works!

schallbert@machine:~# ssh-copy-id -f -i ./ed25519-sk_ssh:server_schallbert_yubikey-main.pub server 
Number of key(s) added: 1

Now try logging into the machine, with: “ssh -i <identityFile> 'server'”
and check to make sure that only the key(s) you wanted were added.

To test it as recommended, log in with the new key and… It works! Only now can the old keys be safely deleted.

How to avoid ssh-copy-id

If you want to transmit the public key without using additional tools, you can copy the contents of the public key file <key-filename>.pub directly into a new line of ~/.ssh/authorized_keys. You just need an open SSH connection to your target.

Thoughts on convenience and usability

When I look at my SSH connections, they broadly fall into two categories:

Admin access (VPS, NAS, Raspberry Pi) and signatures
Protocol access (Git etc.)

In my view, the former require a higher level of security. Furthermore, I don’t use them as frequently as version control systems. I therefore recommend verify-required for the first category and would omit this for the second. I would take the same approach with the no-touch-required option: if a commit is to be uploaded every few minutes, constantly having to touch the key is rather annoying.

All options are displayed when you open the manual using man ssh-keygen. ↩︎

Hardware Security Keys

Schallbert — Sat, 25 Apr 2026

Definitions: TPM and HSK

Acronym	Term	Brief description
TPM	Trusted Platform Module	Soldered securely. Protects a device
HSK	Hardware Security Key	Mobile. Protects accounts, certificates, private keys

The Trusted Platform Module is a security chip that securely stores cryptographic keys and provides functions such as Secure Boot, BitLocker encryption and device identity. It is permanently soldered onto the motherboard of a computer, smartphone, etc.

A Hardware Security Key HSK, on the other hand, is usually portable and can be connected to various devices. It can not only store keys, but often also perform cryptographic operations (key generation, signing, encryption). HSKs are designed to be highly secure and tamper-proof. They can be externally certified or audited, meaning that even highly sensitive processes such as payment transactions can be carried out.

What happened so far

I listened to a podcast from the Bundesamt für Sicherheit in der Informationstechnik (BSI) on the topic of ‘Passkeys instead of passwords’. Following the recommendations, I bought a Hardware Security Key (HSK) so that I can use passkeys for my services in future. For a long time now, I have been unhappy with my fragmented secret management system consisting of notes, browser plugins, offline password managers such as KeePass, SSH credentials stored on my hard drive, backup keys and emergency codes from various providers.

I’d finally like to bring some order to the situation without tying myself so tightly to a commercial provider that I become dependent on them.

Objectives

No longer having to remember passwords or login details
Setting up a system for managing passwords that is as simple as possible, yet secure
Bringing order to the jumble of accounts
Organising my digital legacy
Enabling others to do the same

Welche Erwartung ich jetzt schon einmal nehmen kann: Ganz ohne Passwörter oder Pins geht es auch mit dem HSK nicht. Ein Yubikey ist kein Passwortmanager. Er kann zwar Zertifikate, Passkeys und Secrets für OTPs halten, gibt diese Geheimnisse dann aber nicht mehr preis. Mit seiner Hilfe kann man signieren (Authentizität beweisen), Verschlüsseln und abgeleitete Schlüssel erstellen.

In fact, password management can be centralised and simplified using the Yubikey. Popular password managers support passkeys meaning a master password is no longer required.

Yubikey

As an HSK, I use two Yubikey key fobs. One to carry with me at all times and one as a spare, which is kept in a safe place. There are other providers of mobile Hardware Security Keys, such as Nitrokey. I wasn’t aware of them before writing this article. That’s a shame, as their product is open-source and therefore an ideal candidate for testing on my blog 😥.

Yubico Authenticator

The Yubikey Authenticator GUI programme is useful for setting up the device, assigning functions to the Slot key (e.g. confirmation or password entry via keyboard simulation) and if you use 2FA-Einmalkennwörter z.B. als TOTP. Furthermore, my Yubikey supports PKI, meaning that SSL certificates and private keys can be stored securely.

Concept of the HSK

Part of the concept is that private keys are either generated directly on the hardware key or - without being permanently stored locally - are created on an end device but stored on the Yubikey. These remain on the key and cannot be exported. It supports common asymmetric methods such as RSA and ECDH. For further details, see this BSI publication on cryptographic methods. Furthermore, the key will be kept offline most of the time, i.e. without a power supply. Thus it is virtually impossible to compromise. An additional PIN as a protective measure in the event of key loss is a further security factor.

Spare key

Nevertheless, you should make a note of which access points you protect using which method, so that the correct accounts can be locked in the event of loss. A ‘spare key’ should be available and set up, which is almost as time-consuming as configuring the main key.

NFC

I’d hoped that my key’s NFC support would make logging in even quicker and easier. However, two things bother me here: I simply cannot get my laptop to recognise the Yubikey via NFC. And on my mobile, NFC recognition works at best flakily. When I hold the key up to it, the app launches and asks me to enter the PIN. If I do so, I have to remove the key and hold it up to the sensor again (‘Do not move the key!’). This doesn’t work every time by a long shot and seems rather prone to glitches overall. A pity.

What I find myself missing when using the Yubikey

Overall, I do miss a few convenience features in my day-to-day use of the Yubikey. When I click on the desired keys or accounts, nothing happens. And it seems I can’t configure anything there either.

When using passkeys, I’d like to see a shortcut. Clicking on the relevant key in the authenticator should open the corresponding app or browser window with the target website. Since I’ve already unlocked the HSK with the PIN at this point, the login process can even take place fully automatically once the key is tapped.

I’d like to see something similar when managing accounts. Why is the module called ‘Accounts’ if I can’t access them directly? Here, too, I have to open the relevant web apps etc. myself, have the password manager ready, and then copy the 2FA codes from the authenticator. There must be an easier way.

Touch, password, PIN, PUK, management key, passphrase

Depending on the module, multi-factor authentication is usually required even when using the Yubikey. The simplest method is presence detection via a touch of the dongle. This ensures that a person is actively operating the HSK and cannot, for example, be logged in automatically. Touch can be used across modules; for detailed configuration, see my article on SSH.

The Accounts module can optionally be secured with a password, whilst the Passkeys module is secured via PIN. Some providers even require a PIN to be set up as a second factor.

The Certificates module must be protected against unauthorised access using PIN, PUK and management key. Private keys or certificates can optionally be protected with a passphrase, which locks the keys themselves (and, in my view, makes no sense for the Yubikey, as the private keys cannot be read anyway).

Feature set and cryptographic landscape

Since the HSK promises such a significant security boost, I naturally want to make use of as many of its capabilities as possible. I’ve been playing around with the various modules for a few weeks now and have found that you really need to get your head around them and learn a lot to be able to use them all effectively. At least, once you’ve done so, you’ll have a rough idea of what the modules do and gain a good overview of internet security, encryption methods and protocols.

“Accounts” and “Passkeys” are easy to set up for input forms (platforms, websites, shops) and apps (email clients, password managers, messengers): Highly recommended! - schallbert

Yubikey module	Description
Accounts	Management of accounts/logins, supports TOTP
Passkeys	Passwordless login procedure via FIDO/WebAuthn (public key credential)
Certificates	Storage and use of certificates; supports OpenPGP (PGP) and X.509 (PIV)
Slots	Two programmable slots for generating static passwords or one-time passwords

In my opinion, things get complicated and time-consuming with public/private keys and certificates. Generation and validation procedures depend on the operating system or the programmes available on it, and instructions are sometimes insufficient or contradictory: only for nerds or professionals. - schallbert

Let’s go through the modules.

Two-factor authentication: “Yubikey: Accounts”

The Yubikey can be used as a generator for time-based one-time passwords. To do this, as with any other authenticator app, you create an account for each service and transfer a security key (secret) to the Yubikey. Once the HSK is connected, one-time codes can now be generated based on this secret.

As there is no way to export these secrets from the Yubikey, the process must be repeated for secondary keys. However, not every provider supports the use of multiple authenticators. So it may be sensible to keep the secret in the clipboard¹ and reuse it when configuring the secondary key(s).

Passkeys ‘Yubikey: Passkeys’

The Passkey module on my Yubikey works quite well. Unfortunately, many services² do not yet support passwordless login or only accept Passkeys as a second factor. With other services³, I cannot disable the password, so I see little security benefit, or I am issued with backup codes that are intended to serve as a second factor in an emergency⁴. At least, I can now bypass the password-TOTP process for some services and log in more quickly.

‘The biggest advantages of the technology for users are, on the one hand, that in future they will no longer have to create and manage passwords. On the other hand, passkeys are immune to the vast majority of known phishing attacks.’ - BSI website, 2025 October

You do not need the Authenticator app to configure passkeys. This is done entirely within the browser or the command line. Both must support the handling of passkeys.

Keys & Certificates ‘Yubikey: Certificates’

To shed some light on this, we first need to understand the difference between a key pair and a certificate in the context of authentication. In both cases, the aim is to verify the client’s identity. This may be done with a server (e.g. via SSH), an institution (e.g. a Certificate Authority) or another client (e.g. via PGP). This allows the destination to confirm that the client attempting to connect is indeed authorised to do so.

Public/Private Key Pair

A key pair alone (i.e. without a certificate) is typically used when the communication destination is controlled by the same party as the client. In such cases, the client can place the public key on the host to establish the connection. This method is used, when setting up SSH connections and for certain APIs, such as Google Search Console.

The host can send a test message to the cient encrypted with the public key, which can only be converted back into plain text using the corresponding private key. If the client sends the correct plain text back to the host, this proves to the host that the connection partner is authentic.

Certificates / PIV

But what if the host is not under the client’s control? How can we then ensure that neither the client nor the server is impersonating someone else and has stored a forged public key? This is precisely where certificates are used.

Put simply, the certificate contains not only the public key but also additional information about the network participant and the desired connection. Now a trusted third party, uninvolved in the communication between client and server, comes into play. In the Private Key Infrastructure PKI, this is known as the Certificate Authority CA.

The network participant now sends the public key, together with metadata about themselves, to the CA and submits a so-called CSR, a Certificate Signing Request. The CA then verifies whether the participant’s channel actually belongs to them.

In the case of an email (keyword PGP/GPG), the CA can, for example, send an individual hash link - derived from the requestor’s public key - to the specified email address. If the link is clicked, this proves to the CA that the email address actually belongs to the client, and the CA can then issue the certificate, i.e. sign the CSR with its private key.

For domains (keyword SSL/TLS), the CA can check with the Domain Name Service provider whether the details match the server’s request.

Clients have the signatures of common Certificate Authorities stored (e.g. in the browser). If the certificate is tampered with on the return journey between server and client, its hash value changes and the client displays a certificate warning: ‘invalid certificate’. However, one must now trust the Certificate Authorities⁵ 🙃

The computer should be offline for this. If you want to be extra secure, carry out the pairing on a live operating system. This usually boots from a USB stick and forgets everything after shutdown. ↩︎
E.g. Not at all: Mailbox; 2FA: Discord and Kraken, as of 2026 April. But perhaps I haven’t found the right settings? ↩︎
E.g. Google, GitHub and Hetzner, as of 2026 April. ↩︎
These backup codes must also be stored securely. Ideally offline again, but definitely independently of the HSK. ↩︎
There are alternatives to institutions such as CAs; see Web of Trust. For example, identity verification can also take place peer-to-peer via uninvolved third parties. ↩︎

act_runner rootless: no start

Schallbert — Sat, 28 Mar 2026

The error

My docker-in-docker (DinD) act_runner crashes shortly after starting with the following error message:

Issue ticket on Gitea

I have created a ticket for this issue in the Gitea community (issue #721). Below, I’ll go into my own findings and summarise the discussion a little.

Problem with kernel permissions?

In a Docker-in-Docker configuration, the act_runner must run its own Docker daemon. Only then can the runner create its own containers for the actions. For security reasons, this is not permitted by default.

Why containers are not allowed to start other containers

In the context of act_runner, an action executes code that is itself part of the repository. If malicious code is introduced into the job container unnoticed, e.g. by a ‘collaborator’, it can, in a Docker installation without DinD, under certain circumstances potentially gain direct access to the host system.

Consequences if something goes wrong despite DinD

When using the DinD concept, an attack can result in access to the dockerd process within the act_runner container - and only if the action is not properly encapsulated. Still not ideal, but acceptable: when the container is restarted, the status quo ante is restored. Access to the host system is indirect, as the container itself is equipped with kernel features.

Simple solution: Start the DinD container as `privileged`

To allow the DinD container to set up its own actions, privileged: true can be set in the configuration. This grants the container all kernel capabilities. According to the (excellent) OWASP Security Cheat Sheet, this should be avoided where possible. Should act_runner itself now come under attack or reveal critical security vulnerabilities, the intruder would already have every opportunity to bypass the encapsulation from the host system.

If you want to take the easy route, the corresponding docker-compose.yml file looks as follows.

# docker-compose.yml
# section ACT_RUNNER
runner:
 image: gitea/act_runner:latest-dind-rootless
 container_name: gitea-runner
 privileged: true
# [...]

“And remember: Do not run containers with the –privileged flag!!!” - OWASP’s Docker Security Cheat Sheet, 2026 License

Dead end: Only grant the rights that are absolutely necessary

After a quick search, I find an article on CodeStudy.net¹, which deals with the topic of DinD. From this, I put together some changes to my docker-compose.yml.

My approach:

Make changes to docker-compose.yml
Restart the service docker compose restart <service>
View logs docker logs --tail 100 <container-name>
If started: Run the programme web-app->repo->jobs->rerun_all-jobs
If it fails: Repeat

I start with few, but very powerful, permissions. After a few iterations, I get:

# docker-compose.yml
# section act_runner DinD-rootless
runner:
 image: gitea/act_runner:latest-dind-rootless
 container_name: gitea-runner
 privileged: false
 cap_add:
 - SYS_ADMIN
 security_opt:
 - no-new-privileges:true
 - apparmor:unconfined 
 - systempaths=unconfined
 - writable-cgroups=true
# [...]

But I’m still missing permissions that lie outside kernel privileges and security zones: the Docker daemon needs to be able to access sysfs and proc, i.e. system file directories and processes.

level=warning msg="[rootlesskit:child ] failed to mount sysfs, falling back to read-only mount: operation not permitted"

I cannot resolve this even with cap_add: ALL. Furthermore, online resources on this technical level are really scarce (and I am not a Docker specialist). A re-mount of the file systems might still help here; but that seems too experimental and error-prone to me.

After several more hours of research and trial and error on my server, I have to agree with @tianon in a discussion on GitHub: The Docker daemon requires so many permissions and capabilities that you might as well stick with privileged: true and avoid having to grapple with an armada of CAP_ADD and system bind mounts.

Future solution: Virtualisation or daemonless

However, there may be better solutions: dedicated container runtime environments such as Sysbox. The box itself has no special privileges on the host system; yet, much like in a virtual machine, it appears to be able to provide applications running within it with full access and capabilities.

Covering everything from installation and setup to fully functional job containers would take this post too far afield. Therefore, for the time being I must refer to other sources.

Switching from Docker to Podman is also a possible solution: Podman is daemonless, rootless and offers a similar range of features to Docker. Migrating to Podman would be a project in its own right for me and does not fit within the scope of this post.

Linux Security Modules (seccomp, AppArmor, SELinux)

Definition: LSM enable various security checks and restrictions at the kernel level. Through ‘Mandatory Access Control’, security extensions such as AppArmor can control kernel capabilities for individual applications and block access where necessary.

AppArmor

I run my server (VPS) on Ubuntu. In its more recent versions, this operating system has integrated ‘user namespace creation restrictions’ into AppArmor, which prevents appimages, WebApps and containers from running with elevated privileges.

“Unprivileged user namespaces are a feature in the Linux kernel […]; it enables unprivileged users to gain administrator (root) permissions within a confined environment […]” - mbelair, Ubuntu Discourse, as of May-2026, Ubuntu Discourse Website

In short, this tool was developed as a security-hardening measure to reduce the attack surface on the host system when running programmes that require elevated privileges.

Simple solution: Disable completely

The sledgehammer approach completely disables the feature for user namespaces. We tell AppArmor that third-party programmes may use kernel features or elevated privileges without restrictions, just as in older operating system versions. I found the relevant command on the AskUbuntu forum.

It disables restrictions for this session by overwriting the kernel parameters at runtime (-w).

schallbertTestsThis@machine:~# sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0

I find the command very useful for troubleshooting. If you want to find out whether the desired programme is failing to start because of AppArmor:

Enter the command above
Test the third-party programme, the container, etc.
Reboot. Or enter the command with =1. This will restore the original state.

If you wish to retain this vulnerability permanently, enter:

makeVulnerability@machine:~# echo 'kernel.apparmor_restrict_unprivileged_userns = 0' | sudo tee /etc/sysctl.d/20-apparmor-donotrestrict.conf
makeVulnerability@machine:~# sudo shutdown -r now

This in itself does not constitute a security vulnerability. It has merely become easier, in principle, to exploit any weaknesses in the kernel and escape the container’s “sandbox”.

The correct solution: Tailor settings for each application

There are applications (such as my DinD version of act_runner) that absolutely require elevated privileges to function properly. And only these should be granted the ability to access non-admin user namespaces.

Docker itself has dedicated a section in its documentation to AppAmor. To arrive at the solution for the DinD runner, a bit of transfer (thanks, @thespad) is required. To recap, here is the error message from above:

gitea-runner | [rootlesskit:parent] error: failed to start the child: fork/exec /proc/self/exe: operation not permitted
gitea-runner | s6-svwait: fatal: some services reported permanent failure or their supervisor died

The log file indicates that the parent (Docker daemon in the container) cannot start its child (runner container) because it cannot create processes (/proc/self) for other participants (fork). Who is the user of this daemon? rootlesskit. This is exactly where our solution comes in: we need to enable the AppArmor profile for rootlesskit in the act_runner’s DinD container within the docker-compose.yml file.

# docker-compose.yml
# section act_runner DinD-rootless
runner:
 image: gitea/act_runner:latest-dind-rootless
 container_name: gitea-runner
 privileged: true
 security_opt:
 - apparmor=rootlesskit
 # [...]

The log file now looks fine. Done!

I get the feeling that post was written by an ‘AI’. When it gets specific and talks about ‘critical volume mounts’, there are no actual system paths listed, and the article becomes so vague overall that I can’t actually reach my goal by following the advice. ↩︎

Engrave Stainless Steel A2 / AISI304

Schallbert — Sun, 08 Mar 2026

Why this attempt?

I was asked if I could engrave an auxiliary anchor for a sailing yacht. This anchor is made of 10mm thick stainless steel sheet and, with dimensions of 389x630mm, fits my engraving machine.

I’ve never worked with this material before and was so curious that I accepted the job.

The Material

The anchor is made of AISI 304 stainless steel. Other names for this alloy are ‘18/10’, ‘18/8’ (referring to its chromium/nickel content), or ‘V2A/A2’ (referring to its development from experimental melts or its production process, specifically being “Air-hardened” or by alloy group). It is relatively malleable and therefore frequently used in industrial applications.

However, this stainless steel has a few disadvantages for machining: Its thermal conductivity is quite low. Therefore, little heat is transferred to the material, which causes the milling cutter to get pretty hot. Additionally, the material is tough. The chip doesn’t detach well from the tool, easily creating built-up edges. This results in imperfections in the milling path. If the cutter pushes chips ahead of it, the material isn’t completely cut, and burrs tend to form. Furthermore, stainless steel is work-hardening: if the cutter rubs or dwells on the material, the pressure it exerts alters the material surface, making it harder than the tool itself.

All these properties combined lead to high tool wear and a tight requirements on cooling. The cutting parameters must also be kept in a narrow range.

The Tool

I’m choosing an engraving cutter from vhf specifically designed for machining stainless steel. With a 6mm shank diameter, but only a 4mm cutting diameter, a short straight cutting edge, a 60° engraving angle, and a 0.4mm tip diameter, the tool appears compact and sturdy. The cutter is coated with an AlTiN coating.

Will the thin tip withstand the milling forces and heat generated?

Discussion about milling parameters

In the forum cnczone.nl I’m asking for recommendations on depth of cut, feed rate, and spindle speed for my solid carbide end mill.

My internet research¹ suggests a target cutting speed for AISI 304 of approximately 60m/min and a feed per tooth between 0.02mm and 0.05mm.

I also called the milling cutter manufacturer and asked for recommendations for optimal use on a soft machine like mine. The manufacturer said I absolutely had to use cooling, ideally with plenty of emulsion on the material. His recommendation for the first attempt, using this milling cutter, was:

S15000 F600 Z-0.1 (vhf-Proposal)

I was explained that frequent changes of direction are to be expected when engraving, and the goal should be to guide the machine as smoothly as possible with short acceleration paths. This would largely prevent chatter marks.

From the forum, I received values between F240 and F800 at maximum spindle speed.

I’m incorporating my experience with machining steel from previous articles milling SECC and machining steel, setting a feed per tooth of 0.04mm. I’m adjusting the spindle speed to match the cutting speed of 60m/min. I like to cut quickly through the material, but with a very shallow depth of cut. This results in the following values for the given cutting edge geometry:

S30000 F1200 Z-0.1 (Schallbert)

With this spindle speed and feed rate, I’m exactly double the recommendation from vhf. I choose a fast feed rate to minimize frictional heat at the cutter and transfer more heat to the chip. I keep the depth of cut low to reduce forces acting on the workpiece. This should result in higher engraving accuracy and better detail.

I consider a total engraving depth of 0.2mm enough.

Machine Kinematics

As described above, spindle speed and feed rate are aligned to each other. It is important to check whether the set feed rate can actually be achieved. Depending on the machine’s acceleration capability, as well as the size and shape of the engraving, the machine may, on average, operate at a much lower feed rate due to the numerous changes in direction.

Therefore, the spindle speed and feed rate must be adjusted to the realistically achievable values. These can be observed with the machine in simulation mode. In my case, the machine acceleration is so high relative to the size of the engraving that I can almost always operate at target speed. Want an example calculation?

The machine requires the following time to accelerate from standstill to the set feed rate of F1200:

$$a_{machine} = 400\frac{mm}{s^2}, v_{feedrate} = 1200\frac{mm}{min} = 20\frac{mm}{s}$$$$t = \frac{20}{400} = 50ms$$

So, I only have very short periods where the machine is rubbing against the material more than cutting it.

Preparing the Engraving Process

I only work with metal very rarely. Therefore, I don’t have any ready-made cutting or cooling emulsion on hand. As a substitute, I use Ballistol spray and a little water. I rub the oil into the surface. For the 10x15cm image area, I add about a tablespoon of water.

To my surprise, the two form an emulsion on their own.

I plan to clamp the stock using my vacuum table and position it centrally using locator pins. I close the recesses for handle and stem insert with coated waxed paper (Pro tip! Inexpensive, super tear-resistant, environmentally friendly, almost airtight, easy to work with).

After switching on the vacuum pump, I discover that the armature isn’t sitting completely flat on one side. Instead of -0.7 bar, I’m only getting -0.4 bar. This isn’t critical for the low forces involved in engraving. However, it will be difficult to maintain a consistent line width.

My solution is to scale down the milling design. Due to the smaller size, differences in line width will be hardly noticeable.

Procedure

Before performing the engraving, I run the machine in simulation mode. During this process, I discover two very slow plunge steps. The CAM software likely creates these because of the plunge ramp I set, which it auto-applies to very short lines.

I correct this by manually setting plunge points.

Now I’m running the milling job. The emulsion stays in place despite the spindle fan. The noise level is significantly lower during the first depth of cut than during the second – but it never becomes unpleasant. Vibrations are minimal. Even with the more complex logo, the milling process takes barely five minutes.

Result

I’m really happy. Based on my observations in the forums, I hadn’t expected such a good result on my hobby machine. Virtually no chatter marks, clean milling channels, and only slight burr formation at the stop and return points. However, the material remains in the milling channel and doesn’t create any perceptible edges. Even in detail, the milled finish is quite attractive.

The following video shows some details of the milling process. My last remaining question: Is the slightly rough noise of the milling cutter during the second depth of cut an indication of increased tool wear due to material hardening?

Engraving Stainless Steel↗

Quelle1: VNT Hardware Quelle2: Machiningdoctor Quelle3: CNC Optimization ↩︎

Automate tool length measurement

Schallbert — Tue, 10 Feb 2026

I keep receiving questions about my post on CNC macros. In this article, I’d like to provide some answers and share details on measuring and compensating for different tool lengths.

Question: How does the macro calculate the tool length?

In the section Measuring Tool Length, the following code appears:

1 G53 G00 Z[zSafety] ; # Go to safety height (machine coordinates)
2 G53 G00 X[xPosTls] Y[xPosTls] ; # Go to tool length sensor
3 G53 G00 Z[zSpindleTip + toolLengthEst + 10] ;# Move Z down to 10mm above estimated tool tip

What happens in lines 1-3? If Z=0 is the highest position in the machine coordinate system (move with G53), are zSpindleTip and toolLengthEst negative values?

Answer:

This line moves the machine to the value stored in variable #4506. For me, it is 0, i.e. maximum Z height. This value is permanently stored (see here) and must be set beforehand in the macro setup dialogue.
Executes a movement to the tool length sensor (WZLS) in the machine coordinate system. To do this, I have positioned my sensor at a fixed point on the machine.
zSpindleTip (variable #4509) refers to the height on the Z-axis at which the spindle nose clears the tool length sensor. This value is negative. Why? The maximum value of the Z-axis is zero when it is at the very top of the machine’s travel. If you lower the axis from this point to the tool length sensor (WZLS), you are moving it in the negative Z-direction. If you add the estimated tool length toolLengthEst and a safety margin of 10mm (in case you made a mistake in the tool length estimation), but remain below the mechanical stop of the Z-axis, the value is still negative. However, the spindle nose now has sufficient clearance to the tool tip, because the virtual tool and safety margin have now been added.

Pro tip: In the ‘Variables’ tab of EdingCNC, you can view the values of a variable bank. Enter a bank, e.g. #4500, and you will see the live values of all ten variables.

To understand this properly, it is a good idea to run the CNC software in simulation mode alongside for dry testing with live variables in parallel. Concrete example:

zSafety = 0 ;# machine coordinates, Z-axis at top notch
zSpindleTip = -86.43 ;# machine coordinates, where empty spindle collet touches TLS
toolLengthEst = 35 ;# the length the tool is protruding from spindle collet

G53 G00 Z[-86.43 + 35 +10] # = -41.32mm, so below max Z height

I provide the variable names in my macro repository on GitHub.

Question: How do I calculate the workpiece height after a tool measurement?

The following scenario: You measure the tool and then determine the workpiece height using a macro. If you now change the tool length without calling the tool change macro, the workpiece zero point is no longer correct. With a shorter tool, the zero point is ’too low’; with a longer one, it is ’too high’.

How can this problem be solved?

Answer: Several solutions

In my view, there are three solutions to the problem.

Always run tool length changes through the tool change macro. You can just re-enter the same tool number there.
Reset the Z0 point via workpiece measurement (not recommended¹)
Without a macro: Use the G92 command to shift the Z coordinate system. Example follows.

Solution #3: Shift the coordinate system

You can shift the workpiece coordinate system using the command G92 Coordinate System Offset. When executed, the number passed for the selected axis is adopted as the new value in the workpiece coordinate system.

In this case, we only want to shift the relative position and thus compensate for the difference in tool lengths.

Let’s assume that the old tool T17 has a free length of 32mm and the new tool (again T17, as in this example we are simply changing tools or using a finishing tool with the same cutting data) has a free length of 24mm. The new value is shorter, and therefore the zero point must be moved down by 8mm. ‘Downwards’ is a negative value on the Z-axis in the coordinate system.

;# Pseudocode
;# (each CamelCaseWord represents a variable ID like #5003):
G92 Z[CurrentPositionZ + (OldToolLength - NewToolLength)]
;# 20mm + (32mm - 24mm ) = 20mm + 8mm = 28mm

Offset of workpiece zero vs. workpiece coordinate system

This calculation may seem confusing at first, as it shifts the coordinate system upwards. But this is precisely what is required to maintain the workpiece zero point relative to the tool tip: the tool is 8mm shorter. Since we do not shift the workpiece zero point downwards, the coordinate system must be shifted upwards. This compensates for the now longer distance between the tool tip and the workpiece.

Coordinate System	Tool tip position Z [mm]	Workpiece position Z [mm]	Tool top position Z [mm]	Workpiece position Z [mm]
	Before shift	Before shift	After shift	After shift
Workpiece	20	0	28	0
Machine	-41.42	-61,42	-41.42	-69,42

Note that shifting the coordinate system changes the position of the tool tip in the workpiece coordinate system without the Z-axis actually moving. The machine coordinate system remains unaffected by this change. However, as the Z-axis now has to travel a greater distance to reach the workpiece, its position in the machine coordinate system is lower by -8mm.

Prerequisites

The macros only work under certain conditions:

The tool length has been measured beforehand, or tool changes are always performed through the macro.
The workpiece zero point has been determined with known tool length.
There hasn’t been any emergency stop or other critical error in between. This would clear the volatile memory for the relevant variables.

In my view, re-measuring the workpiece height in the Z-axis after a tool change adjusts the wrong parameter. In reality, it is not the height of the workpiece that has changed, but the position of the tool tip. I would therefore always prefer to adjust the parameter that has actually changed. ↩︎

Steel sheet (SECC) milling

Schallbert — Mon, 22 Dec 2025

The Project

This is a PC case made of 0.8mm thick, galvanized steel sheet “SECC”. It dates from the early 2000s and was recently fitted with new internal components.

The Challenge

Due to the now much more powerful graphics card (TDP 200W) and the cramped design as an HTPC, the resulting heat loss could not be dissipated as desired, resulting in temperatures around 50°C inside the case under load. That’s far too high for my liking.

The Specifications

The graphics card follows current, common designs: A metal radiator covers the entire surface of the PCB. Its fins are oriented perpendicular to the socket, and the flat fans mounted on it (3x80mm) push the air through the baffles toward the motherboard and case lid.

Unfortunately, there are only a few millimeters of space between the graphics card and the case lid. Therefore, I want to add extra holes to the lid at precisely this point. The case already has 3mm diameter ventilation holes in a 10x19mm 45° grid pattern on the sides. I will now try to apply this pattern to the ventilation holes for the graphics card. This brings me to a total of 133, which I’d rather not mark, center-punch, and machine by hand for convenience.

So I create a drawing in CAD and then a drilling cycle program for my milling machine.

Tool 1: 3mm Drill Bit

I do have a few hardware store steel drill bits made of standard high-speed steel (HSS) lying around, but some of them are in poor condition. So I use an unused steel drill bit from my Dremel set.

Drilling Parameters

It has a light gray, metallic color and looks so similar to uncoated solid carbide tools that I (mistakenly) assume it is carbide. Therefore, I selected the following parameters:

3mm solid carbide drill bit for steel, 2 flutes: S17000 F470 Z-0.2, retraction 0.5mm

As usual, I work completely dry, meaning I don’t use any coolant.

Drilling Preparation

To prepare the workpiece, I apply masking tape to the section of the housing where the holes will later be drilled. Using the maximum drilling depth, I only penetrate the material minimally. Since the masking tape remains intact this way, I don’t lose vacuum through exposed holes in the vacuum table.

I place the housing cover, outer side down, on the machine bed and cover any unused holes in my vacuum table with a rubber mat. Then I position the gantry close to the holes so that the spindle doesn’t collide with the housing wall during the machining path. I also remove the move to park position G28 from the milling file, as the housing cover is now in the way. After the Z-axis is raised from the last hole, the program ends with M30 (stop and rewind).

The Result

To cut to the chase: The drill bit didn’t last. I noticed something was wrong after just a few seconds from the sound of the machine: It was suddenly much quieter than before.

I managed a measly 3 out of 133 holes this way. After that, the tip of the drill bit was stuck in one of the partially completed holes and seemed to be welded to it.

Problem Analysis

Apparently, the drill bit wasn’t made of solid carbide at all, and therefore I was using completely unsuitable parameters. I posted the symptoms in two forums (Hobbyline and cnczone) and received the information that I should have been using S1000 F30 (!) if I wanted to drill a few more holes.

The Solution

So I’m switching to carbide end mills. I only have one in my collection that was made for steel. However, it has a larger diameter, so I have to adjust my design.

Now I’m creating slots in CAD, which are 6mm wide and 22mm long, positioned at a 45° angle above the graphics card radiator.

Tool 2: 6mm Solid Carbide End Mill

I’m applying my already proven milling technique for SECC and refining it further: High spindle speed, high feed rate, shallow depth of cut.

6mm solid carbide end mill, AlTiN-coated, 2 flutes, 0.2mm 45° chamfer: S10600 F880 Z-0.18

Cutting SECC steel of my PC case↗

Cut Quality

The cut edges are burr-free, high-gloss, and razor-sharp. Therefore, a chamfer needs to be applied. I do this by hand with a scraper and utility knife, which I quickly regret: I can’t get it as clean and even as I’d like.

Not so bad: I get a neatly finished case with significantly improved cooling practically for free (apart from the learning curve). In the image above, the graphics card’s radiator is clearly visible through the new slots in the case.

Jekyll-dockerimage: Bundler and Gemfile

Schallbert — Fri, 14 Nov 2025

What is bundler?

Bundler is a tool that can be used to manage and version dependencies between modules and libraries for Ruby language. I use Bundler commands regularly, e.g. to build my blog and put it live on the server.

If I want to build locally, I type

bundle exec jekyll serve --incremental --future

into the console. This command tells Bundler to run the jekyll application in server mode and provides it with parameters that prevent jekyll from completely rebuilding every time a file is changed and still create pages that have not yet been published.

bundler in the CI pipeline

I also use Bundler as part of a Jekyll Docker image to publish my site, as I have often linked to, e.g. when moving to self-hosted. Since switching to a new version of the blog software lately, I have been seeing puzzling errors in my CI pipeline. Locally however, the system builds flawlessly. I have documented how to address and fix such errors here.

“There was an error while trying to write to /path/to/Gemfile.lock” - CI console output

No write permissions

Like the Jekyll user, Bundler itself can only read files on my CI and cannot write them. I can fix this for the Jekyll output with chown, but I don’t want to allow bundler to do this: I require Gemfile.lock to remain identical between my local build environment and the CI so that I can fix errors in advance.

Specification file for bundler

The Gemfile.lock file contains all dependencies of the application used, including version numbers and sources. If only the source code of the application plus this file is provided, bundler can pull a specific version of all dependencies during build time, allowing to work with similar requirements regardless of machine and version.

Troubleshooting

A first attempt to fix the issue by aligning all versions between local and CI failed. The second attempt, to give the Jekyll user write permissions to the Gemfile.lock, also failed. The third attempt led me to the Bundler website, where I took a closer look at the parameters, keyword “frozen”.

The fact is that you can prohibit bundler from rewriting the Gemfile.lock. To do this, use the command

bundle config set frozen true

According to its documentation, processing is aborted as soon as the file is about to be rewritten.

Although I had added this command to my yaml file in the CI, the build failed again. The trigger was the same library nokogiri as before, but the error message was now much more helpful:

Optimize `Gemfile.lock` for different environments

So I follow the suggestion and execute the desired command on my local machine:

bundle lock --add-platform x86_64_musl # my CI runner's environment

Note: musl is a libc-implementation for Linux.

My lockfile now has the following entry:

nokogiri (1.18.10-x86_64-linux-musl)

With this error fixed, I get better portability of my web page creation setup as a side effect.

fail2ban: Configuration error

Schallbert — Thu, 23 Oct 2025

Every few weeks, I usually check my server:

Is everything OK with Docker? docker ps
Are the backups being created correctly? borgmatic repo-info
Are my defenses working? iptables -n -L
Are my update automation daemons running? htop -F /server-config

This time, however, iptables showed a blank page on the DOCKER-USER chain. This is unusual and indicates that something is wrong with fail2ban.

schallbert@machine:~# iptables -n -L

Chain FORWARD (policy DROP)
target prot opt source destination 
DOCKER-USER all -- 0.0.0.0/0 0.0.0.0/0 
DOCKER-FORWARD all -- 0.0.0.0/0 0.0.0.0/0 

Chain DOCKER-USER (1 references)
target port opt source destination

The output reads something like

“The FORWARD chain forwards, as required by Docker, to the container-relevant DOCKER-USER chain. However, no applications are listed under this chain that can reject or prohibit packets from being forwarded (REJECT/DROP).”

Two entries for caddy-server and caddy-ratelimit should appear here.

“fail2ban ERROR Failed during configuration”

So I first check Docker:

docker logs -t fail2ban # Print logs for the container named fail2ban with a timestamp

Instead of being rewarded with <timestamp> Server ready after scrolling through as usual, I see a never-ending flood of log messages. They repeat themselves and show that fail2ban is exiting with Exit with code 255. I get a better overview of the log if I only print the last 100 lines:

docker logs -t --tail 100 fail2ban # return latest 100 lines of log with timestamp

The error message

<timestamp, ID> ERROR Failed during configuration: Bad value substitution:
option 'action' in section 'caddy-ratelimit' contains an interpolation key 'banaction' which is not a valid option name.
Raw value: '%(action_)s'

This message is interesting because I had problems with jail.local at exactly this point when I was creating the article Fail2ban with Caddy. At the time, I had to remove action = iptables-multiport from the configuration because the triggering IP addresses had landed on the wrong CHAIN (INPUT).

Now, the default action on the DOCKER-USER chain seems to no longer work. So, I’m searching a few sources for a possible solution. However, I can’t find any references to updates that would cause major changes in the action.d folder either in the update history of fail2ban or in the linuxserver.io distribution I’m using.

Solution via `banaction = iptables[type=multiport]`

So I tried the web search and found a comment from a fail2ban maintainer:

“[…] With action = iptables[type=multiport, protocol=tcp, chain=DOCKER-USER] you overwrote the default action […] [Solution:] simply set the chain variable only in the jail […]: port = http, https banaction = iptables[type=multiport] chain = DOCKER-USER” - sebres, Dec-2024

The basis for this statement can be found in the Docker Docs. Side note: In all places, it is recommended to use the more modern nftables instead of iptables for future applications. Here is an article on the topic.

This allows me to completely delete the action field that was previously present in my jail.local and use banaction instead. It now looks like this:

# jail.local update Oct-2025

# Server/Client errors: 401 403 404 500
[caddy-status]
enabled = true
chain = DOCKER-USER
port = http,https
filter = caddy-status
logpath = <path/to/logfiles>
findtime = 10min
maxretry = 5
bantime = 1d
banaction = iptables[type=multiport]

With this configuration, fail2ban boots normally and displays Server ready.

Verifying the solution using iptables

As described above, I create entries in the test log files that should trigger the actions of caddy-status and caddy-ratelimit. As expected, I get the following in the “blocklist”:

schallbert@machine:~# iptables -n -L

Chain DOCKER-USER (1 references)
target prot opt source destination 
f2b-caddy-ratelimit tcp -- 0.0.0.0/0 0.0.0.0/0 multiport dports 80,443
f2b-caddy-status tcp -- 0.0.0.0/0 0.0.0.0/0 multiport dports 80,443

Milling Albizia/Albasia

Schallbert — Mon, 06 Oct 2025

What is Albasia?

Albasia wood (Falcataria falcata), also known as Albizia, is a very light softwood. It is found primarily in Southeast Asia and grows rapidly, which improves its sustainability rating. Its color varies between whitish-yellow and pastel brown-pink, but is significantly lighter than beech or oak.

In Europe, it is usually sold as plywood. This mitigates disadvantages such as the strong tendency to shrink and warp.

The density of Albasia is about 15% lower than that of the already very light poplar wood. It forms long fibers as it grows, which impairs its machinability. Because it also provides good insulation due to its numerous air pockets, it is often used in vehicle interiors (campers, boats, etc.). To compensate for its low strength, Albasia multiplex is often offered as a sandwich panel rather than raw.

Sandwich technology achieves good material properties

My test material has a 0.8mm thick layer of high-pressure laminate (HPL) applied on both sides. The surface is available in colored melamine decors and can be ordered with a hardened finish, which makes it matte and scratch-resistant.

Are Albasia composite panels suitable for outdoor use?

“Our panels are not intended for outdoor use.” - EpicPLY, Retailer

Even though the material was developed for interior construction, I’m currently testing the weather resistance of the panels with an experiment. I’m screwing the composite panels together at right angles, butting them together so that one side of the coating and the other side of the wood edge are directly exposed to the elements. I’ve sealed one exposed edge with hard oil and left the other untreated. Now I’m hanging the structure outside on the balcony for a year and will then see how the material behaves – an update will follow in a separate post!

Setup for weathering and screw mounting tests with Albizia plywood.

Vorbereitung für den Schraubentest

Der zusammengesetzte Testaufbau ist zur Bewitterung angebracht.

Am Tag 0: Beide Oberflächen sehen noch frisch aus.

My expectation: unlike birch plywood and similar to poplar wood, Albasia will not be suitable for outdoor use. I assume that the fibers in the edge area will absorb water and cause the material to swell. This would quickly lead to its decay. Furthermore, I consider the risk of insect infestation to be high because the fibers are really soft and can be easily peeled off. Therefore, I would only use it indoors.

Screw connections

Despite its light core, the Albasia composite panel is easy to screw together and, with a “screw pull-out strength of 120kg” for a 4mm wood screw, is in a “very stable range”, according to my dealer.

As a layperson I can’t make much sense of such figures, so I conduct a small experiment. I take a 4x35mm wood screw and look at the specified tightening torques. For my manufacturer, the following applies:

Parameter	Value
Pull-out parameter $ f_{ax,k} $	14 N/mm²
Torsional strength $ f_{tor,k} $	3 Nm
Screw-in torque $ R_{tor,mean} $	$ \frac{f_{tor,k}}{R_{tor,mean}} $ ≥ 1.5
Pre-drilling in softwood	2.5 mm
Pre-drilling in hardwood	3 mm

Source

Now I’m checking if I can butt-join two boards with the specified tightening torque. For this, I’m using a torque screwdriver. But I don’t even get to the point where the mechanism engages: the screw starts to pull itself through the surface layer before that happens. I estimate the maximum torque before the screw “disappears” into the material to be only 0.8Nm.

Now I’m checking if I can butt-join two boards with the specified tightening torque.

Your browser does not support the video tag.

Video: Trying to tighten a screw to the correct torque in wood.

This method certainly allows for stable screw connections, but I would always proceed carefully when tightening, use a generous number of screws, and additionally choose dowel or form-fitting connection methods.

Albasia on the CNC router

Albasia can be processed with a CNC router like any other softwood. Problems typical of softwoods, such as tear-outs or fiber clusters, are to be expected here as well. In addition, the hard coating makes machining even more difficult due to different parameter selection requirements.

Which router bits are suitable for machining?

The material is challenging for the router bit. I had already had bad experiences with milling HPL in the article about overloading my router motor. However, the material at that time had a much higher proportion of hard paper fibers compared to Albasia composite (chips were light brown and interspersed with many “dots”), which blunted the router bit in record time.

I use solid carbide router bits with chip breaker and minimal helix (15°) to process the long wood fibers. Otherwise, they quickly clog the extraction system and accumulate around the router bit, which can pose a fire hazard.

For edge processing, I’ve had good experiences with compressive router bits like the quarter round bit. If you only plunge slightly into the material, a V-cut is also suitable. With a greater plunge depth, half-round “ball-nose” router bits achieve a nice milling finish.

The image shows a comparison. On the right, the raw material has a green-gray coating, cut on a panel saw. On the left, in purple, is the workpiece I CNC-milled. The cutting pattern is clearly different and the surface machined with the milling cutter is much cleaner and without any tears.

If you’re using chipbreaking cutters in the full groove, it’s worth adding a finishing pass with a straight-toothed cutter and high feed rates with only a small infeed of 0.2mm.

All cutters must have the following in common:

a cutting edge as sharp as possible (no coating!)
a high cutting speed (don’t use diameters that are too small)
a relatively high feed rate.

I was able to achieve a high surface quality with single-flute cutters. The large chip channel ensures good removal, but chipping can sometimes be problematic when single-passing with full engagement. The cutting load is quite high.

Which parameters?

Albasia is so soft that it offers little resistance to the milling machine’s axis drives. The cutting speed should be set high; online sources recommend around 500m/min as a guideline. I’ve tried feed rates up to 6000mm/min. As I only machined small workpieces, even higher feed rates would hardly offer any time advantage due to the constant changes in direction. An additional finishing pass is mandatory due to chipbreaker marks.

I’ve noted a good compromise between milling time and surface quality for various cutters below. However, I only use the engraving cutters in the HPL layer, which is why I reduce the feed and speed slightly.

Tool (s = number of cutting edges)	Feed rate F [mm/min]	Speed S [rpm]	Z feed [mm]	XY feed [mm]
Solid carbide 2s chipbreaking 15° 6mm	4200	24000	12	2.7
Solid carbide 2s straight 6mm	3700	26500	21	0.2
Solid carbide V-cut 3s 90° 10mm	3000	21000	0.27	0.1
Solid carbide radius cutter 2s 6mm	3000	21000	0.27	0.1
Solid carbide quarter round 4s r2 6mm	3600	15000	2	2

Finishing pass

The following video shows the finishing process after roughing a 145x145mm workpiece. Due to the small lateral infeed, hardly any fiber strings form, and the low feed rate ensures a very high surface quality despite the slight twist of the cutter.

Your browser does not support the video tag.

CNC finishing of Albasia composite material

Engraving

With coated Albasia panels, engraving is ideally done in the top layer. If the material is not penetrating the substrate, problems with tearing and loose fibers can be avoided. Since the top layer is not very thick, the choice quickly falls on V-shaped cutters with an obtuse angle or a flattened tip.

Alternatively, pockets of 3D engravings can be cleared with end mills, if the CAM program offers such an option. This creates completely smooth “floors”. Alternatively, radius cutters can be used due to their gently rising geometry in the lower area. If the CAM program does not support 3D engraving with radius cutters, the cutter geometry can be approximated as described in the article Halftone Images on Dibond.

I have prepared two examples in the image below: The CR code in the upper half was created with a 6mm radius cutter at a 0.2mm plunge depth. The resulting dots and lines feel soft and pleasant thanks to the smooth transitions. The lettering in the lower half of the image was milled with an engraving cutter and a depth of up to 7mm. While this beautifully showcases the natural wood tone, small letter heights like this (max. 30mm) tend to cause breakouts in letters with “islands” (a, e, b, o, etc.).

Working with Color

The contrast of the engravings is OK, but the legibility is far from perfect. Therefore, it makes sense to ink the engravings. I experimented with a few colors; you can see some impressions in the image below.

Which varnish?

The board comes with a protective film to protect the colored coating. This makes it super easy to apply spray paint if I leave the protective film on the workpiece for engraving. I used standard water-based acrylic varnish (top right), spray paint with metallic particles (below), and a solvent-based permanent marker (top left).

The varnishing process was easy. The abrasion resistance of the surfaces engraved in HPL is high with color applied. If you engrave through to the wood, the fibers will stand up slightly due to the varnishing, which is very time-consuming to correct later and not beneficial to the overall impression.

Please note: Protective film

The varnishing process should be started immediately after engraving, if possible, and not days or even weeks later. Otherwise, the protective film tends to peel off or bubble at the edges of the engraving, and the color is drawn into areas where it doesn’t belong by capillary action. Unfortunately, this visibly ruined the gold-blue version of my test series.

Conclusion: What is Albasia for?

All in all, I would consider Albasia to be one of the more docile materials for CNC machining, accepting a wide range of feed parameters while maintaining a high surface quality. A challenge is its tendency to form long fibers. Due to the gluing in the multiplex composite, poor quality of individual inner layers can become apparent late in the processing, which then leads to scrap and time wasted.

It’s also suitable as an engraving material, but the coating alone would be sufficient for this, and the small engraving depths place strict limits. Therefore, the engraving can at best be a nice addition to the actual use of the panels.

The coating makes the material quite suitable for furniture construction. If you work with concealed edges, I can even imagine it being used in the kitchen. Because it has high flexural stability and the coating’s impressive hardness, it’s a surface suitable for everyday use. The panel material is feather-light, making it ideal for mobile applications in visible areas.

However, it’s nowhere near as robust as birch plywood. I wouldn’t trust the material to withstand stressed joints; for bolted connections, I would maximize surface area and always use furniture connectors (i.e., avoid wood screws).

passtdas - a helper for your CNC

Schallbert — Mon, 08 Sep 2025

The Problem

A friend of mine wrote to me. He uses a CNC gantry milling machine as a hobby, just like me, and often produces unique pieces. However, the resulting leftovers are sometimes too valuable to throw away. To maximize material utilization, he tries to use these offcuts as blanks for future projects.

“Do you know how I can find out on the machine whether my milled part fits on the blank?” - Anonymous

This happens: According to the dimensions, the workpiece just fits on the blank. If you set the zero point even slightly incorrectly, the milling cutter will move beyond the workpiece boundaries, and scrap it is.

The Solution

It would be great if the machine followed the contour of the workpiece before starting the milling process, wouldn’t it? This way, you could easily find out whether the blank is actually completely within the workpiece boundaries.

Since I can’t find a quick solution online right away, I’m thinking about writing my own software to implement this. So I’m getting to work.

TODO: ADD VIDEO LINK

The Software

For a quick prototype, I choose the high-level language python. Why? I have a fairly good command of python, libraries for the necessary geometric calculations are available, and the written program can be easily compiled into an all-in-one file.

The program is designed to read machine instructions (G-code) and analyze the milling paths. The positions at which the maximum and minimum X and Y values are approached in the milling program are to be saved and output as a motion command in a path file.

If this program is loaded onto the machine, an “extreme value” determined in this way is approached and the Z-axis is lowered from the safety height to the measuring height. The program is then paused so that the operator can make adjustments to the workpiece position. Only by entering another start command will the next extreme value be reached, and so on.

This program will be called passtdas.

Program type

I’m choosing a console program with few parameters:

Shortcut	Parameter name	Description
`-f`	`--file`	The source file to be analyzed. Required.
`-s`	`--zsafety`	Safety height the machine will do flyovers with. Optional.
`-p`	`--zprobe`	Probe height the machine will pinpoint to. Optional.

The two height parameters are given default values, so they don’t always have to be specified.

Input

passtdas reads files and searches them line by line for motion commands like G00, G01, G02, G03. While the analysis is simple for linear movements like G00, G01 – linear movements can be expressed using point-to-point connections whose extreme values cannot lie between the points – it can be quite challenging for circular segments.

The height information on the Z-axis is also read and saved. This allows the machining technician to quickly check whether the maximum milling depth matches the desired one.

If the G-code file contains no instructions or incorrectly formulated instructions, passtdas displays corresponding error messages and aborts the analysis.

Definition of Circular Segments

Background: Circular segments can be defined in two ways in G-code.

[...]
G00 X5.660 Y-0.000 Z10.000
G02 X-5.660 Y0.000 R5.660 F1600
[...]

Command:

“Move a circular arc clockwise from point X5.66 Y0 with radius 5.66mm to point X-5.66 Y0 with feed rate 1600mm/min at height 10mm.”

Drawing these commands creates a hanging semicircle. Circles can be defined using three points. However, only two points are given. The third point must be calculated from the specified arc radius using the start and end points.

Task: Find the extreme points of this arc in the XY plane.

But an arc can also be defined like this:

[...]
G00 X-2.5696 Y5.0431
G03 X-4.6258 Y3.2616 Z-0.5000 I2.5696 J-5.0431
[...]

Command:

“Traverse a circular arc counterclockwise from point X-2.56 Y5.04 with center coordinates I2.56 J-5.04 to point X-4.62 Y3.26.”

Drawing these commands creates a short circular segment in the upper left quadrant. Although three points (start and end points, as well as the center of the circle) are given, it can sometimes be difficult to determine whether an extreme value is being reached somewhere along the arc.

Task: Find the extreme points of this circle in the XY range.

Calculating the extreme points of circle segments

A circle segment can have up to four additional extreme points, depending on its range: X+, Y+, X-, Y-. For a full circle, these are always the coordinate intersection points as seen from the radius.

def get_extremes_from_arc(arc, coordinates):
 # [...]
 x_plus_radius = [center_x + radius, center_y, xyz[2]]
 y_plus_radius = [center_x, center_y + radius, xyz[2]]
 x_minus_radius = [center_x - radius, center_y, xyz[2]]
 y_minus_radius = [center_x, center_y - radius, xyz[2]]
 extremevalue_order = [x_plus_radius, y_plus_radius, x_minus_radius, y_minus_radius]
 if (arc[1] - arc[0]) < 0:
 # crossing the 0° line (x-axis), handle overflow with nested if below
 arc[1] += 360
 result = []
 for crossing_angle in range(0, 721, 90):
 if crossing_angle in range(int(arc[0]), int(arc[1] + 1)):
 i = int(crossing_angle / 90)
 if i > 3:
 i -= 4
 result.append(extremevalue_order[i])
 # [...]

If the circle center and radius are known, the swept angle can be determined from the start and end points. This makes it clear whether the circle’s start and/or end points themselves represent extreme values, or whether additional extreme values arise from sweeping over a coordinate origin. It is precisely what the above code example does.

Data table

All possible extreme values (two for a line, up to four for a circle segment) are now collected in a data table. Once each G-code line has been evaluated and the list is thus complete, the maximum and minimum values for each axis are determined from this list.

This ultimately creates four XYZ coordinates, which describe the maximum extent of the workpiece for each spatial direction.

At the same time, the program searches all commands to the Z-axis and remembers the maximum plunge depth.

Output

The command-line program has no visual output. Therefore, I’ll limit myself to creating a simulation program for the milling machine in which the workpiece extension is approached and paused at each point:

G90

# Go to coordinate zero
MSG "Zmin of this job: -1.0"
G00 Z40
G00 X0 Y0
G01 Z15 F1200

MSG "PathPreview: Hit START to go to Ymin: ['0.0', '-5.66']"
M00
G00 Z40
G00 X0.0 Y-5.66
G01 Z15 F1200

MSG "PathPreview: Hit START to go to Xmin: ['-5.66', '0.0']"
M00
G00 Z40
G00 X-5.66 Y0.0
G01 Z15 F1200

MSG "PathPreview: Hit START to go to Ymax: ['0.0', '5.66']"
M00
G00 Z40
G00 X0.0 Y5.66
G01 Z15 F1200

MSG "PathPreview: Hit START to go to Xmax: ['5.66', '0.0']"
M00
G00 Z40
G00 X5.66 Y0.0
G01 Z15 F1200
# [...]

The above example traverses the extreme values of a circle with r=5.66mm. It always moves to a safety height of Z40 to cover the points and, upon arrival, slowly moves the Z axis at F1200 to the target height of Z15. It is important that the zero point in Z has been correctly measured beforehand – otherwise, a collision with the workpiece may occur.

Testing and Verification

To see if the program works as expected, I created three test files:

Unit tests: Tests individual functions in the program, e.g., whether line and circle segments are correctly recognized
Integration tests: Tests whether functions function correctly in conjunction, e.g., the correct calculation of extreme values based on input values
End-to-end tests: These tests check the entire function chain, e.g., whether incorrectly formatted input files are recognized

Finally, I create several test files with the program and move to the extreme values generated from them with my CNC.

Open-source software

The source code for passtdas is available for free on GitHub, but must be interpreted using Python.

I’ll build a user-friendly and easy-to-use program once enough people have expressed a desire for it.

Rootless act_runner for Gitea

Schallbert — Sun, 10 Aug 2025

In this article, I’ll show how to change the act_runner of my Gitea instance from gitea/act_runner to gitea/act_runner:latest-dind-rootless.

Why all this effort? Because act_runner runs in Docker and requires access to the daemon via /var/run/docker.sock as a so-called volume (meaning a disk) or bind mount to function. The owner of this socket is root, which gives the container virtually full access to the host system.

“RULE #1 - Do not expose the Docker daemon socket (even to the containers)” - OWASP / Docker Security Cheat Sheet

This is highly risky, because it’s inherent in the runner’s design to execute job code generated by third parties. It’s an open gateway. In conjunction with direct access to the host system, this could result in a total failure or hostile takeover of my infrastructure in the event of a successful attack.

Danger from `docker.sock`

In this guide, I follow the OWASP’s recommendation on Docker security and apply its rule(s) in practice. OWASP stands for “Open Web Application Security Project.” It is an organization dedicated to improving the security of web applications and supporting users like me with free articles, documentation, and technology.

The measures described under Rule 1 are:

Leave Docker’s tcp socket disabled

If access to the Docker Daemon is enabled via tcp, it can be connected to via an unsecured connection and without authentication - unless further precautions have been taken. The daemon is then accessible to virtually any internet user and thus vulnerable.

So how do I ensure that the tcp socket is disabled? The instructions for this can be found in the Docker Docs and must be applied in reverse. There, too, there is an explicit warning against opening the tcp socket unprotected.

After completing all the necessary steps, I run netstat once. It lists open sockets, network interfaces, and routing tables:

# check if Docker Daemon's "dockerd" tcp socket is exposed
schallbert@machine:~# netstat -lntp | grep dockerd
# test is a pass if this command does not return anything.

I don’t find any entry with dockerd. My server isn’t vulnerable at this point.

Do not include the Docker socket `/var/run/docker.sock` in other containers

This is where things get a bit more complicated: act_runner needs the socket to create, manage, and ultimately dispose of job containers. Without access to the Docker Daemon via the socket, the build pipeline simply won’t work – unless you want to forgo Docker entirely and run both runner and the build jobs directly on the host machine. This comes with many disadvantages: loss of encapsulation, lack of portability, poor scalability, reduced security…

But there’s also a solution for the Docker option: Setting up act_runner Docker-in-Docker. With this setup, act_runner receives its own Docker Daemon, but with limited permissions and without access to the host system. This then takes over the lifecycle of job containers so that they run completely independently of the host system.

Docker in Docker

The following diagram illustrates the difference:

Starting Act_runner “rootless”

So I follow the instructions and copy together a suitable docker-compose.yml. Important here:

privileged: true must be set. Otherwise, the Docker daemon in the act_runner container cannot start properly because it lacks kernel functions. This causes the entire container to crash repeatedly without generating any helpful error messages.
The environment variable DOCKER_HOST=unix:///var/run/user/1000/docker.sock must be set. Here, the Docker socket is controlled by a non-privileged user and is available to the runner for managing job containers. The daemon runs encapsulated in the container and is not connected to the host machine.

As of Jul-2025: Problem with 0.2.12-dind-rootless

This is where I encountered my first problem. The runner crashes shortly after starting with the following error message:

schallbert@machine:~# docker logs gitea-runner
# [...]
[rootlesskit:parent] error: failed to start the child: fork/exec /proc/self/exe: operation not permitted

Help came from the Gitea community. act_runner runs smoothly with the previous version 0.2.11. However, for reasons unknown to me, it is displayed on Gitea as 0.2.12.

Volume Confusion

Great! Now that the runner is working, I’ll let it run a job right away. Unfortunately, the build fails after just a fraction of a second with this message:

# Runner step: Set up job
failed to start container: Error response from daemon: error while creating mount source path '<volumeSourceFullPath>': mkdir <volumeSourcePath>: permission denied

I had to research this for hours and was under the false impression for a long time that it was due to insufficient permissions on the folders on the host machine. Only later did I truly understand that Docker-in-Docker means exactly what it says: Not only are containers created by containers, but a separate Docker daemon runs within the container!

This means that the classic method of making volumes in job containers available directly from the host system using -v /a/b:/x/y no longer works. Instead, volumes must now be passed through. Example:

host directory /opt/server/www/blog-artifacts -> act_runner volume /tmp/blog-artifacts –> job container volume /tmp/blog-artifacts

The :z in the act_runner volume is now important. It indicates to Docker that this volume is shared between containers. These volumes must not only be specified in act_runner’s docker-compose.yml, but the job scripts in the .gitea/workflows/ folder must also be adjusted accordingly. However, the :z on the “right side” is not needed here.

“is not a valid volume”

Despite all my efforts, my jobs still aren’t running. This time, because of an error message that already seemed familiar:

# Runner step: Set up job
[/tmp/blog-artifacts] is not a valid volume, will be ignored

So, go into the config.yml of act_runner and add the volume names:

# /gitea/runner/config.yml
# [...]
valid_volumes: ["/tmp/blog-artifacts", "/tmp/lectures-artifacts"]
# [...]

In this file, I can leave privileged: false because, unlike act_runner, the job container doesn’t require kernel features.

Set permissions correctly

Now I’m getting Permission Denied error messages again when running my jobs, although not directly in the first step of the actions. Since I’ve now checked the volume paths down to the last detail, it can only be due to the folder permissions on the host machine.

In order for the artifacts created by the job container to be stored via the volumes on my host, I have to pass the directory to be written and all subfolders -R to the previously defined, non-privileged user ID=1000:

schallbert@machine:~# chown -R 1000:1000 /target/path/to/artifact/

Finally, everything is working smoothly, and I’ve put a stop to the (unlikely, but possible) takeover of my host system by malicious job containers.

Add rate limiter to Gitea

Schallbert — Fri, 25 Jul 2025

Here, I’m providing a step-by-step guide to solving my Gitea crashes problem. I’m incorporating the experience I gained from using a rate limiter for my blog with the goal of making the application more robust and protected against undirected denial of service attacks.

Step 1: Specify the log source for the rate limiter

Gitea naturally generates logs itself, theoretically down to the access level by external clients. However, in the past, I haven’t been able to export Gitea’s access logs from Docker. Therefore, we now instruct Caddyserver, which is connected as a reverse proxy, to create logs on behalf of Gitea. Access to the Gitea web interface will then appear in these logs.

In the Caddyfile, the log module looks completely unspectacular.

# /caddy/Caddyfile

#[...]
log {
output file /log/gitea/access.log
}

Exclude internal traffic from the log

Now I look at the logs and see entries that I don’t want. For example, act_runner generates a Fetch Task POST to Gitea every two seconds and a GET request every ten seconds with the same target as a Health check. These don’t even need to appear in the log for me. My first thought here was to simply not log requests from internal IP addresses. However, since Caddy acts as a reverse proxy, all IP addresses run “internally” without exception.

So I have to recognize act_runner-specific logs differently:

# /caddy/Caddyfile

#[...]
git.schallbert.de {

 reverse_proxy * http://gitea:3000

 # Enable logging for fail2ban, don't log for runner
 log_skip /api/actions*
 log {

 output file /log/gitea/access.log
 }
}

The log_skip directive now instructs caddy to no longer create logs for access to /api/actions*.

Reverse Proxy: Displaying Remote IPs

However, I still have a problem: If all requests in the log come from an internal IP address, how am I supposed to block “bad” requests? A web search shows that this is a common problem for reverse proxies in Docker. Unfortunately, many solutions cannot be applied to this situation because they use different server software like nginx or have different services running behind their proxies than Gitea.

But it’s actually quite simple: Just insert a line in the correct place in the Caddyfile, and the remote IP addresses come in unchanged.

# /caddy/Caddyfile

#[...]
git.schallbert.de {
 reverse_proxy * http://gitea:3000 {
 trusted_proxies 172.16.0.0/12 # Docker-internal netwock traffic runs with these IPs
 }
#[...]

The trusted_proxies directive tells Caddy that the network address range provided by Docker can be trusted. Now the actual source IP address is shown instead of the container’s internal interface address. This is exactly what I need to be able to analyze the addresses later with fail2ban.

Step 2: Determine the Use Case

Let’s look at how many HTTP 200 ok requests are coming in the edge case between normal use and “abuse.” To do this, I surf around Gitea and click on a bunch of things that I would never normally do at that speed as a human. Then I analyze the logs.

This gives us some initial benchmarks for the rate limiter.

# rate limiter tests
findtime = 10s
maxretry = 10
bantime = 6h

Step 3: Configure Fail2ban

Next, we configure the filter and jail file of fail2ban to analyze the logs defined above.

Filter

Here, I’m re-using the same filter for my rate limiter from the previous article. We’re only interested in successful requests that we count within a time window.

Jail

I’m using the values from the rate limiter tests 1:1 in the jail file. I’m referring to caddy-ratelimit (link above) as the filter. It is essential to select the DOCKER-USER chain as the requests are routed via Docker’s virtual network.

# /fail2ban/config/fail2ban/jail.local
# [...]

[gitea-ratelimit]
enabled = true
chain = DOCKER-USER
port = http,https
filter = caddy-ratelimit
logpath = /var/log/caddy2/gitea/access.log
findtime = 10s
maxretry = 10
bantime = 6h

I reboot fail2ban to activate the protection.

Step 4: Test the rate limiter

I proceed exactly as in my article fail2ban with caddy and get the same result. It works!

Rate limiter with Caddy and fail2ban

Schallbert — Thu, 10 Jul 2025

Here I describe how to configure fail2ban (commonly used to defend against unauthorized access attempts) to limit the number of successful accesses within a defined time window. This is called “rate limiting” and is intended to thwart denial of service attacks that overload my server.

Enabling Caddy Logs

First, we need to enable the web server’s logger. With Caddy, this can be done with just a few lines of code that I add to my existing configuration:

# caddy2/config/Caddyfile

blog.schallbert.de {
# Define webserver
root * /www/blog
encode gzip
file_server

# Enable logging for fail2ban
log {
output file /log/blog/access.log
}

# [...]

From now on, logs will be created in the specified path, which I manage with a logrotate configuration.

Configuring Fail2ban as a Rate Limiter

To recap: fail2ban accesses the machine’s packet filter rules and, in effect, modifies its firewall to ward off dynamic attacks. For this to work, fail2ban must be provided with log files generated by a host application, such as a web server, that contain the IP addresses of the client computers.

Preliminary consideration: “http 200 OK” as a filter?

Even normal, permitted accesses 200 OK must count for my rate limiter.

To get a feel for how my logger records accesses, I take a look at the logs. I get the number of entries that contain a status of 200 OK returned. To do this, I perform a search with grep and calculate the number of hits line by line using wc -l.

schallbert@machine:/var/log/caddy2/blog# grep -o '"status":200' access.log | wc -l
847

Okay, so there have already been 847 hits today, which received a 200 OK response. Now I’ll call up my blog’s landing page and display the article overview.

schallbert@machine:/var/log/caddy2/blog# grep -o '"status":200' access.log | wc -l
858 // after calling blog.schallbert.de
schallbert@machine:/var/log/caddy2/blog# grep -o '"status":200' access.log | wc -l
882 // page loaded after clicking "Posts" button

Wow, that was 35 entries for two clicks! Looking at the log, it explains the many lines of assets—images and logos—that are being loaded.

Unfortunately, this makes it clear that the filter criterion status:200 for my rate limiter can’t work without further ado. The number of notifications depends largely on the respective article. So, I would have to define a threshold above which no normal person would generate access to my blog.

In order to count only “real” accesses to my pages, I have to somehow exclude the assets from the logs. Fortunately, there’s a simple directive in Caddy for this.

# caddy2/config/Caddyfile

 # Enable logging for fail2ban
 log_skip /assets*
 log {
 output file /log/blog/access.log
 }

log_skip now ensures that all accesses to files in the assets/ folder and below are not logged.

The behavior of Fail2ban is defined using two configuration files:

Directory `filter.d`

filter.d/<filter-name>.conf contains the definition of an event to be monitored. Thus, the failregex “page not found” http 404 status code can be used to trigger the packet filter, just as successful access can be used to set up my rate limiter, which responds to 200 OK. I copied the regex from Rafael Kassner.

# /opt/fail2ban/config/filter.d/caddy-ratelimit.conf

[Definition]
failregex = "client_ip":"<HOST>"(.*)"status":200
datepattern = \d+
ignoreregex =

In addition, further boundary conditions are defined in the file, e.g., the date format datepattern is adapted to the log output of the system to be protected.

`jail.local` file

jail.local determines the conditions under which the packet filter for the client IP becomes active and blocks further access attempts. For use as a rate limiter, I need the following fields:

findtime - the time window in which attacks are counted, maxretry is the number of permissible attempts in the time window, and bantime is the time for which the attack is blocked. ignoreip is usually preconfigured to the relevant internal IP addresses by default. In my case, act_runner, for example, sends a message to gitea internally to query whether any new automation tasks are pending. I definitely don’t want to interfere with these accesses.

The example for fail2ban used below shows the rate limiter on my blog. In the end, the files and log entries for gitea are essentially the same. Only the filter names and parameters differ due to the different requirements of the website and the DevOps platform.

# /opt/fail2ban/jail.local
# ...

[caddy-ratelimit]
enabled = true
chain = DOCKER-USER
action = iptables-multiport
port = http,https
filter = caddy-ratelimit
logpath = /var/log/caddy2/blog/access.log
findtime = 20s
maxretry = 10
bantime = 6h

Once these two files are configured, you can restart fail2ban and view the logs. If there are still errors in the files, fail2ban will output the following:

ERROR Errors in jail 'caddy-ratelimit'. Skipping...

If this occurs, for example, the jail cannot be assigned to the filter (must have the same name!), a field definition such as maxretry is incorrectly typed, or a number format cannot be read. If everything is working correctly, the entry will be:

Reading config files: /etc/fail2ban/filter.d/caddy-ratelimit.conf

Testing the Rate Limiter

Now I want to check if everything is working as expected.

Checking the Filter Regex

The first step is to look at the filter term and check whether it can be reliably found in the logs. Conveniently, fail2ban has a built-in tool for this: fail2ban-regex <logfile> <filter>. So I can simply enter the filter file and a test log file and see if I get any matches.

fail2ban-regex ./access.log /etc/fail2ban/filter.d/caddy-ratelimit.conf 

Running tests
=============

Use filter file : caddy-ratelimit, basedir: /etc/fail2ban
Use datepattern : \d+ : \d+
Use log file : ./access.log.2
Use encoding : UTF-8

Results
=======

Failregex: 2009 total
|- #) [# of hits] regular expression
| 1) [2009] "client_ip":"<HOST>"(.*)"status":200
`-

Ignoreregex: 0 total

Date template hits:
|- [# of hits] date format
| [2685] \d+
`-

Lines: 2685 lines, 0 ignored, 2009 matched, 676 missed
[processed in 0.12 sec]

Quantitatively, I get what I expect: the search procedure described above using grep gives me exactly the same results.

Check Jail

The next step is to test whether the rate limiter is working. To do this, I simulate log entries. I significantly ease jail rules for this test, otherwise I’ll have to generate too many entries in a short period of time.

# /opt/fail2ban/jail.local TEST
# ...

[...]
findtime = 10s
maxretry = 2
bantime = 100s

After restarting fail2ban, I create a second console where I can simulate log entries. This must include at least the time, IP address of the caller, and status; however, I play it safe and use complete entries.

echo '{"ts":1751246139,"remote_ip":"1.1.1.1","status":200,[superLongIrrelevantOtherStuffForFiltering]}' >> /var/log/caddy/blog/access.log

I enter this multiple times to exceed my limit of 2 retries.

In the first console instance, I then check the fail2ban status for the corresponding filter.

fail2ban-client status caddy-ratelimit
Status for the jail: caddy-ratelimit
|- Filter
| |- Currently failed: 0
| |- Total failed: 0
| `- File list: /var/log/caddy2/blog/access.log
`- Actions
|- Currently banned: 0
|- Total banned: 0
`- Banned IP list:

That was nothing. The fail2ban log file reveals why:

# /fail2ban/config/log/fail2ban/fail2ban.log
WARN [caddy-ratelimit] Ignoring all log entries older than 20s; # probably messages generated within a fail2ban restart period

So again with appropriate timestamps. I have to get these from the current system time ($(date +%s.%N)) if I have to constantly adjust it. Therefore, I take a complete log line, modify it to the IP address 1.1.1.1, and insert appropriate timestamps:

 echo "{\"level\":\"info\",\"ts\":$(date +%s.%N),\"logger\":\"http.log.access.log0\",\"msg\":\"handled request\",\"request\":{\"remote_ip\":\"1.1.1.1\",\"remote_port\":\"38229\",\"client_ip\":\"1.1.1.1\",\"proto\":\"HTTP/1.1\",\"method\":\"GET\",\"host\":\"blog.schallbert.de\",\"uri\":\"/vacuum-clamping/\",\"headers\":{\"Accept-Encoding\":[\"gzip, deflate, br\"],\"Connection\":[\"keep-alive\"],\"User-Agent\":[\"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0\"],\"Accept\":[\"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8\"],\"Accept-Language\":[\"en-US,en;q=0.5\"]},\"tls\":{\"resumed\":false,\"version\":772,\"cipher_suite\":4865,\"proto\":\"http/1.1\",\"server_name\":\"blog.schallbert.de\"}},\"bytes_read\":0,\"user_id\":\"\",\"duration\":0.001207354,\"size\":12290,\"status\":200,\"resp_headers\":{\"Last-Modified\":[\"Tue, 10 Jun 2025 19:33:01 GMT\"],\"Content-Encoding\":[\"gzip\"],\"Server\":[\"Caddy\"],\"Alt-Svc\":[\"h3=\\\":443\\\"; ma=2592000\"],\"Vary\":[\"Accept-Encoding\"],\"Etag\":[\"\\\"gzip\\\"\"],\"Content-Type\":[\"text/html; charset=utf-8\"]}}" >> access.log

After calling this command several times, I now get

Status for the jail: caddy-blog-ratelimit
|- Filter
| |- Currently failed:	1
| |- Total failed:	11
| `- File list:	/var/log/caddy2/blog/access.log
`- Actions
 |- Currently banned:	1
 |- Total banned:	4
 `- Banned IP list:	1.1.1.1

That looks great.

Check packet filter

Phew, finally done! This is how fail2ban.log looks like right now. It shows [caddy-ratelimit] entries as expected:

schallbert@machine:~# nano /fail2ban/config/log/fail2ban/fail2ban.log

 <timestamp> <id> INFO [caddy-ratelimit] Found <IP a>
 <timestamp> <id> INFO [caddy-status] Found <IP b>
 <timestamp> <id> INFO [caddy-ratelimit] Ignore 172.18.0.1 by ip
 <timestamp> <id> INFO [caddy-status] Ignore 172.18.0.1 by ip
 <timestamp> <id> INFO [caddy-ratelimit] Found <IP c>
 <timestamp> <id> INFO [caddy-ratelimit] Found <IP c>
 <timestamp> <id> INFO [caddy-status] Found <IP c>
 <timestamp> <id> INFO [caddy-status] Found <IP c>
 <timestamp> <id> INFO [caddy-ratelimit] Found <IP d>
 <timestamp> <id> INFO [caddy-status] Found <IP d>
 <timestamp> <id> INFO [caddy-status] Found <IP e>
 <timestamp> <id> INFO [caddy-ratelimit] Found <IP e>
 <timestamp> <id> INFO [caddy-ratelimit] Found <IP f>

To test the entire chain, we now need to check whether the corresponding IP address is actually blocked in the hardware. To do this, I enter:

schallbert@machine:~# iptables -n -L | grep "1.1.1.1"
REJECT all -- 1.1.1.1 0.0.0.0/0 reject-with icmp-port-unreachable

Everything seems to be fine. For one final test from an external IP address, I call a Link Checker in the browser, which should crawl my website and thus trigger the rate limiter. Although it crawls 44 links in a very short time, the test is displayed as “passed.” Very strange, [caddy-ratelimit] should have triggered!

iptable-chain INPUT instead of FORWARD

Confused, I take a look at the entire iptable. At the same time, I look up the IP address of the link checker. It appears in the iptables. And yet, it’s apparently not blocked. Why is that? A closer look shows:

Chain INPUT (policy ACCEPT)
target prot opt source destination
f2b-caddy-ratelimit tcp -- 0.0.0.0/0 0.0.0.0/0 multiport dports 22

The problem here is that my jail is on the INPUT chain. However, the requests don’t go directly to my server hardware, but are forwarded via Docker to Caddyserver. To work, I have to land on the FORWARD chain, where DOCKER-USER already is. Very strange, since I had specifically specified chain = DOCKER-USER in jail.local. Something must be overriding this definition.

An inconspicuous forum post together with the multiport comment in the iptables printout led me to the solution: The action = iptables-multiport directive overrides my chain = DOCKER-USER statement, because the following is set in the corresponding configuration file iptables.conf: chain = INPUT.

So I simply delete the action entry, so that fail2ban reverts to the default for DOCKER-USER: multiport dports 80,443.

Now the jail name also appears correctly in iptables.

schallbert@machine:~# iptables -n -L
Chain DOCKER-USER (1 references)
target prot opt source destination
f2b-caddy-ratelimit tcp -- 0.0.0.0/0 0.0.0.0/0 multiport dports 80,443

Finally, another link check shows that my rate limiter is working.

Gitea crashes: Too many requests

Schallbert — Mon, 30 Jun 2025

Here I’d like to briefly describe how a service crash repeatedly paralyzed my server for hours. So badly that I could only restart it from the provider’s console. I’ll explain how it happened and how I intend to avoid this and similar problems in the future.

I was attacked. Or was I?

I was working on an article that I wanted to post later. To be on the safe side in terms of backup, I created a commit as usual and wanted to push it to my Gitea instance. But my git push command simply didn’t work.

Confused, I tried to access my website. The response was:

Strange. Then I wanted to log in to my server to check everything was OK: ssh <servername>. Again, the terminal remained unresponsive. Bummer!

Timeout. As a last resort, I logged in to my hoster’s web interface and looked at the server’s graphs:

Oh, what’s going on? I’m trying to shut down the server via the web interface. That doesn’t work either. Only a hard reboot succeeds. I can log in again via SSH and see that all Docker containers are booting up normally.

What happened?

It’s a good thing I keep all the logs for a week. This, along with the machine’s utilization over time, allows me to reconstruct what happened. At least to some extent.

Which logs help?

Localize the crash! Find the triggering application in the kern.log and note the timestamp.
Are there system-wide effects or other services being affected? Check the syslog.
If you suspect the system may have been hacked, auth.log has the details.
If the affected application is running in a container, the relevant logs there may be helpful.
Review the application’s logs. The time just before the crash is particularly interesting.

And here again, in detail, are the logs I reviewed for my behavior and where they can be found.

log name	purpose	relevant content for this issue
/var/log/syslog	System-wide (bare metal) messages	(docker) warning: “health check for container `<ID>` timeout” (containerd) error: “ttrpc: received message on inactive stream”
/var/log/auth.log	Contains authentication messages from external hosts, in my case mostly SSH	None
/var/log/kern.log	Logs all app, service, daemon and system crashes	Out of memory: Killed process `<ID>` (`<serviceName>`)
[.]/gitea/log/gitea.log	Protocol for requests, actions on Gitea’s web interface, repo changes etc.	`GET` requests, crash/restart indications
docker container logs `<containerID>`	Docker’s logs for the container in question	Received signal 15; terminating. (SIGTERM)

kern.log/syslog

Here, next to the stack trace, I can see exactly what happened. The most understandable message is the following:

Out of memory: Killed process `<ID>` (gitea)

So, Gitea was unplugged because it had consumed practically all system resources. If I scroll up in the log, I see a few minutes earlier:

<timestamp> <machineName> dockerd[716]: level=warning msg="Health check for container <ID>"
<timestamp> <machineName> dockerd[716]: level=error msg="stream copy error: reading from a closed fifo"

Already here, I see warnings that the gitea container isn’t working properly. Another advantage of including “health checks” in the docker-compose.yml file. For other reasons, I’ve already written an article about the purpose and implementation of health checks in docker.

gitea.log

The following log entry indicates that Gitea has just been restarted due to a bug in a submodule:

cmd/web.go:205:serveInstalled() [W] Table system_setting Column version db default is , struct default is 1

And further up in the log:

Very interesting. The highlight.css is in my public repo. It’s the scheme for lectures.schallbert.de’s appearance. This file is quite large, almost 1 MiB. And it’s loaded here dozens of times, practically for every commit.

Now I’m looking at other crashes in the past. It’s always bursts of GET commands for large files or requests for compares between two branches of the repository that precede my server crashing.

Who’s behind this?

All crash-triggering requests come from the same IP address range. The crashes started a few weeks ago. But mostly at times of day that I(and apparently many of my readers) didn’t notice. And after a few minutes, the server was always back to normal operation.

Oh, the trail leads to Google’s Cloud Platform (GCP).

But I had blocked robots?

Indeed, I had stopped the search engine indexing for Gitea. Strange. Or am I not dealing with one of the Google spiders or “AI” scrapers, but with a nasty hacker who rented a virtual machine from “Google Cloud”?

Log research: How frequently is the file requested?

A spider would only crawl all my pages once every few weeks, right? And hopefully not ignore my robots.txt. A crawler certainly wouldn’t make the same request multiple times and at short intervals, would it?

To check this, I search the Gitea logs for entries of GET requests to one of the large and therefore resource-intensive files to transfer:

gunzip gitea.log.<date.rotateID>
grep "<filename>" gitea.log.<date.rotateID>

In the list, I see that the same request to the same target file is made multiple times from the same IP address, all within seconds. Shortly before the crash, my server took almost 4 seconds to serve the request.

I also see that the IP address changes every few hours.

Countering a DoS Attack

In summary, I have to conclude that I’m being attacked via a Denial-of-service attack from the GCP address space.

To gather a bit more background information, I visit a few websites on the topic. There, I learn that gitea on my server is crashing under an Application Layer Attack. Now that I know what’s happening and what the problem is, finding solutions is much easier 😅

Of course, I don’t want to give up without a fight by permanently taking my Gitea instance offline. So, what options do I have?

Provide more resources

Admittedly, my machine only has 40GB of memory and 2GB of VRAM, as well as a measly 2-core CPU from 2009. I could book a more powerful server to better handle peak loads. But this wouldn’t prevent the attack, only mitigate its effects.

Rate limiting directly in the web server

Rate limiters limit the number of requests a client can make within a certain time window. This conserves server resources. This is a “soft” defense against DOS attacks, as triggering IP addresses are briefly and gently blocked with an error message. Typically, HTTP status code 429 “Too Many Requests” is returned when the limiter intervenes.

Gitea doesn’t have a rate limiter. In my setup, Gitea runs behind a reverse proxy provided by my Caddyserver. So, that’s where I’d have to start. For Caddy, rate limiters are only available as external modules, which must be manually installed and configured in xcaddy.

Blocking with IP-table rules

Here, you could again use fail2ban and simply block multiple requests for the same resource from an IP address. Gitea has a description of the setup in its documentation. In my case, I would have to continue where I left off, and thus monitor not only SSH but also normal page requests.

This solution sounds the most sensible to me, as it clearly separates concerns into different applications. I only use applications I already have available: Caddy would provide the access logs, and fail2ban would have to read them and set filters in the jail.local configuration so that it acts like a rate limiter.

Let me link the follow-up article Setting up Fail2ban with Caddy here 🙂

My own webshop - Part3

Schallbert — Sun, 15 Jun 2025

Selecting a solution based on key questions

Which solution is straightforward and quick to set up?
Which solution is easier to maintain?
With which system can I, as the operator, most easily understand how everything works (transparency, dependencies)?
Which system has the smallest attack surface?

There’s still quite a bit missing here!

Over the last few weeks, my enthusiasm for setting up my own online shop has waned a bit, as I wanted to focus first on securing my server and finishing off some old, nearly completed projects. I’ll return to this series as soon as that ‘I-want-to-do-something-new’ fever takes hold of me again.

Hang in there! Schallbert

Creating an own Webshop - Part 2"

Schallbert — Thu, 15 May 2025

A few options for webshop software

Let’s assume only software that at least partially meets my requirements get shortlisted. Below, I would like to present a few possible ecommerce solutions.

Shopify

Shopify is an all-inclusive package solution for online retail. Shopify Inc. is headquartered in Canada and handles its European business from Ireland. The Legal sections found on the website, as well as the complexity of the information presented on the topic of Data Protection, lead me to suspect that Shopify is a multinational corporation – and not just a provider of online shops.

But back to the topic.

The domain and hosting are included in the price with Shopify, and you can create your own shop rapidly and without any programming knowledge in a browser window by selecting a theme, and setting a few parameters. Shopify has extensive documentation and offers training and all sorts of other services for its subscribers.

But where’s the fun in that? After all, I run a tech blog and am not a businessman, literally. A bit of programming is definitely something I enjoy. Furthermore, as an open source fan, I don’t particularly like the corporate stance (just look at their modern website, equipped with auto-playing videos and optimized for marketing purposes). The configurability is mediocre at best, and, as with most other solutions, I have to do the optimization for search engine results myself. Regarding data protection and security, they maintain the usual compliance and state-of-the-art communication; so I can’t look behind it and have to assume that they protect their customers well, also in their own interest.

Shopify describes itself as a closed-source SaaS platform.

Therefore, I’m removing this platform from my list of preferred web shop solutions because I don’t want to become unnecessarily dependent on the well-being of third parties. With Shopify, it’s not me who has control over subscription costs or “my” share of the cloud, but Shopify. It’s not me who can determine how many transactions are included in the “plan,” but Shopify. It’s not my customers who remain in control of their data, but Shopify or a third-party company they have commissioned to process it.

Wordpress / WooCommerce

I only know Wordpress as a juggernaut. The jack of all trades, capable of everything you could possibly want thanks to an overwhelming mass of plugins, extensions, and features. And much more you don’t need.

But it’s open source, free, and, according to its own statements, used by many, many shop operators. Let’s take a look at the most well-known e-commerce extension for Wordpress: WooCommerce.

I immediately found countless guides on setting up a WooCommerce store (for example, from GeeksForGeeks or Themeisle), so I’ll refrain from providing my own guide.

After skimming through a few tutorials, I’ll summarize:

A WooCommerce store is easy to set up and configure using a wizard.
There are many themes available, and easy-to-integrate store designs offer a suitable look for every taste.
It helps to already have a WordPress site and some basic knowledge of domains, DNS, and hosting.
Many payment service providers and some shipping providers are preconfigured.
There are numerous YouTube tutorial videos on the topic, and detailed documentation is also available.
Countless plugins allow you to optimize your store for even very specialized areas and purposes.
However, this variety of options can be offset by the fact that choosing the “best solution” can be difficult, and optimizing the site for speed and search engines can become complex.

All in all, WooCommerce strikes me as a balanced mix of “I want to quickly put together something useful” and “I want to be able to configure and modify everything.”

Snipcart

I would consider Snipcart a “shopping plugin” for any website. Even static websites that can essentially only display a product catalog become a fully-fledged eCommerce solution with Snipcart. The core element here is the shopping cart, which is used to process the purchase.

The interface is the website’s HTML markup. Snipcart is integrated as a JavaScript blob and referenced via HTML. The design and appearance are handled via cascading style sheets. In my case, I could use the existing “tech stack” consisting of a static Jekyll site with a CD pipeline in Gitea on a Caddyserver instance for the shop as well, and with minimal adjustments, conjure a seamless look for the blog and shop. Sounds great, right?

Snipcart is developed by a Canadian company in Quebec. It describes itself as a “Shopping Cart Platform”. An account is required for installation, and you must enter your shopping domain in the Snipcart web frontend, similar to Google Search Console or Google Analytics, to receive a so-called API key. Only with this key will the JavaScript blob integrate correctly.

And this brings us to the first major problem: Snipcart restricts my freedom.

<link rel="preconnect" href="<https://app.snipcart.com>" />
<link rel="preconnect" href="<https://cdn.snipcart.com>" />

This shows me that Snipcart would like to keep the data on its server. I have to use their content delivery network to handle the order process and integrate it with mine. So, it’s by no means the case that all connections remain with me and I “buy” the JS code from Snipcart and run it in the customer’s browser or a database solution behind it on my server.

So, if full autonomy and control over your own site isn’t a priority, Snipcart can be a very elegant e-commerce solution. This is especially worthwhile for existing sites. However, for me, it’s out of the question.

OpenCart

OpenCart offers a PHP-based shop module free of charge and open source. It can be booked as “Software as a Service” through third-party providers or downloaded via Github and deployed on your own server.

Installation and administration are handled via web interface and, in my opinion, are straightforward. You can also find a few images on DockerHub that you can get started with right away.

OpenCart is fully customizable to your needs. Templated themes can also be integrated. Out of curiosity, I visited a few websites that use OpenCart: In my opinion, the shop system is well-structured and easy to use. However, the loading speed of the sample pages isn’t always super fast.

Overall, I consider OpenCart, with its customizable, lightweight shop solution in PHP, to be a good solution - also thanks to its still-active community (13+ years!).

Django

Django is a web framework for Python. It offers many features, allows extensions by simply including Python modules, and allows even beginners to quickly create working solutions. Although there are numerous libraries for websites, shops such as django-SHOP, database integrations, integration of payment systems, etc., everything still has to be put together and configured by the developer.

While this gives you full control and a solution that’s perfectly tailored to your needs, it also has a significant drawback: learning time. Database security must be ensured by the developer, and user input validation must be programmed or at least correctly configured. Customer master data is stored in plain text by default. Access and deletion concepts must be created and implemented by the developer, or time should be allowed to identify and integrate suitable modules.

If frameworks are to be used, it is recommended to check for active community maintenance beforehand. For example, it looks to me as if there hasn’t been much activity on the Github repo for django-SHOP since 2021. A quick search, however, reveals a lot of alternatives - now it’s time to consider something suitable or reinvent the wheel yourself 😃

The alternative for beginners like me: no shop system

What is meant by “no shop system”? A form -based approach. Here the user looks at a static website that contains catalog and product pages. In addition, a few elements are shown: E.g. a drop-down menu for variant and a field for the number of pieces. Next to it a simple button. If this is clicked, a form opens that automatically transmits the clicked content and asks about the user’s contact details.

If everything is filled in properly, the form creates a PGP-encrypted email to me when sending. I can then view the order analogously to manually created order emails, send an order confirmation and request payment.

With a few products, a shop equivalent can be implemented supporting a few customers without much effort. Of course not as comfortable as with an eCommerce solution. But maybe that’s enough for the beginning.

In the next article

Selection and Implementation

Creating an own Webshop - Part 1

Schallbert — Tue, 15 Apr 2025

This is part one of a series of articles about building my own webshop. I’m listing my requirements and wishes for my e-commerce solution. This way I narrow it down to a few suitable software solutions that I could later use for my webshop.

What my webshop needs to be able to do

For me as owner, the webshop should operate as lean, transparent, and automated as possible. I want to spend little time on operation, maintenance, and support. Payment processing, order status management, customer registration, sending customer information, and a large number of steps in the shipping process, such as postage and entering shipping confirmations, should be automated or at least prepared for automation.

But the webshop must also be structured in a way that is understandable for customers: simple and with just a few clicks from the homepage to checkout, easy to use, similar to other common webshops, with straightforward payment processing. As little data as possible should be collected from the customer.

My webshop only needs to display a few products. Therefore, navigation can be kept very simple. Due to the small number of product categories, a tree structure may be completely avoidable.

Let’s take a detailed look at what features the web shop ultimately needs to provide.

Functions for Admins

Create a product

Title
Tags
Short description
Description
Images
Videos
Price
Profile / Data sheet / Instructions

Edit a product
Remove a product
Manage orders

Confirm a step
Restart a step
Cancel a process

Manage customers

Create (sandbox)
Delete (data protection)

Functions for Buyers

Browse the webshop
Configure product(s) or select product variants (e.g., color, quality)
Add product(s) to a shopping cart
Manage shopping cart
Register (optional, only for customer data storage and order management)
Select shipping
Make a payment

Automatic functions

Payment registration
Invoice creation and dispatch
Inventory management
Shipping notification

Non-functional Requirements

Here I list everything the web shop should be able to do “on the side.” Things that are not directly required for order processing or visible to customers.

Usability on the Operator Side

The shop should have as few dependencies on third-party software as possible and only integrate those modules that are absolutely necessary. Examples:

Payment processing
Shopping cart
Customer account / address data
Inventory
Order processing (mailer) for buyers
Invoicing.

The shop should only contain as many dynamically generated elements as absolutely necessary.

Usability on the Buyer Side

The web shop should feel like current, common e-commerce solutions: nimble, clear, simple.
No obligation to have a customer account.
The shop should support common payment methods (debit, credit, PayPal, Apple/Google Pay, etc.). Only local (German-based) payment service providers are acceptable.
Inventory levels and production times should be displayed transparently for the user.

Data Protection

The shop should only collect data necessary for business transactions.
If technically and legally possible, the shop should be able to operate without a cookie banner.
Customer accounts should be automatically deleted after two years of non-use (with two weeks’ notice).

Look & Feel

Here are some examples of successful, simple shop designs:

Requirements derived from this:

The web shop’s structure and color scheme should match my Blog page and provide a consistent look.
Only static fonts should be used without reloading.
Product descriptions should be able to be uploaded as Markdown documents (or similar).

Database or file solution?

It doesn’t always have to be a database. For many applications, only a fraction of the functionality of common database solutions is required, but this significantly increases the complexity of the overall system. For example, if concurrent writing rarely or never occurs and there is no strong data concatenation, a file solution can be simpler, faster, and more efficient Linking a related “Engineering Kiosk” podcast episode.

On the other hand, many shop systems come with batteries included, meaning they already have built-in data management that is almost transparent to the programmers.

File solution (static website)

From the operator’s perspective, the web shop should feel like a Git project (everything is a file).
Changes to the offer, item descriptions, prices, etc. should be able to be pushed to the server via git push – just like with a static website.
If a pull request is merged into ‘main’, the web shop should be rebuilt (CD pipeline) and the changes published to the server.
Dynamic elements are required for inventory management, shopping carts, customer interaction, and automated functions. Here, it is important to decide which elements are best handled directly in JavaScript and which are better handled via an automation pipeline.

Database solution (dynamic website)

Products are stored in the database.
The web shop is managed via a web interface.
Database management should be partially automated, with backups and encryption for customer data provided.
The database is transparent for the admin: There is no direct interaction; everything is channeled through e.g. a web interface.

In the next article

Comparing e-commerce solutions for my webshop

PeerTube instance migration

Schallbert — Tue, 01 Apr 2025

Brief definition

Fediverse?

Fediverse calls something Fediverse when several providers use the same protocol to allow users of these providers to interact freely with each other on digital media. This creates highly interconnected but decentralized services, where free choice of provider is part of the concept. The Verge goes into detail here for your reference.

PeerTube?

PeerTube is a decentralized video platform where multiple servers share the load and viewers can also help distribute videos. Roughly speaking, PeerTube is to YouTube what Mastodon is to X.

The Reason

This article begins with an email.

“You’re receiving this non-automated email because you’re one of the handful of people who has a video hosted on this PeerTube instance. I have decided to shut down this service […]” - Tom, scitech.video administrator

He was really sorry, but continuing to operate the instance would cost too much time, maintenance, and money. Furthermore, the platform is currently underused, so the effort seems disproportionate.

So far, so understandable.

Tom’s Story: scitech.video

Since I had uploaded and linked all my videos there, I was very interested in the background. So I asked for more information. My questions were answered comprehensively and promptly.

The Beginnings

Tom set up his PeerTube instance in 2019. Originally to share his own content. Shortly thereafter, he opened the hosting service to others, albeit with some unease.

From an admin perspective, such a video platform seems like the “Wild West”. Ensuring healthy growth for its offerings has proven quite difficult, especially because of the many bots and advertisers involved.

“Growing the service turned out to be challenging. Every time I opened registrations spambots started creating accounts easily and uploading videos with advertisements in the video descriptions […]” - Tom, scitech.video administrator, via E-Mail

Lots of bots and unwanted users around

While he was always able to prevent these unwanted spammers, scammers, and advertisers from contaminating scitech.video, this still meant constant work and manual monitoring of the automatic rejection and moderation tools.

At this point, I’d like to agree with Tom. I have an average of about 340 failed connections from external IP addresses to my server per day, which I then block. On the ssh port alone. And my server is neither particularly large nor well-known, nor can it be considered a worthwhile target. Not to mention the countless attempts to access my Gitea user interface, crawl the website for hidden links, and much more.

For operators of a portal that also grants external users write access (videos, descriptions, tags, comments), the problem is even more serious. Unlike my site, participation is part of the concept here.

Copyright?

Exactly. From the platform operator’s perspective, every video is fundamentally subject to copyright or related rights. But is the user who wants to publish this video also the author? Or has this person acquired usage rights?

Depending on the country and the associated legal situation, administrators must address this issue very carefully. High penalties can be imposed on both administrators and users if a copyright infringement occurs.

The former, at least in Germany, can be resolved according to a Federal Court of Justice ruling of October 23, 2024, as follows: “[…] after a clear indication of a violation of law, the content posted there must be checked for similar violations within the scope of what is technically and economically reasonable, and infringing content must be blocked or deleted.”

However, due to the authorities’ strong focus on specific subject areas, violations are usually quickly noticed – because users can also “report” posts.

Some countries are currently preparing laws to protect children and young people from using social networks and the associated dangers. For reference, here is a link to the Federal Agency for Youth and Media Protection. Here, too, the administrators are required to implement these rules – but how?

At least, this is what Tom is asking himself, given the increasingly diverse legal interpretations of different countries.

Updates and Maintenance

“PeerTube upgrades were a bit of a drag because everything was tightly coupled with their specific nginx configuration, with breaking changes that had to be adjusted every couple of minor versions.” - Tom, scitech.video administrator, via email

The work isn’t particularly difficult to perform, but it does take a lot of time. As with other software, it can happen that a PeerTube server won’t boot after an operating system update, for example, or that incompatibilities arise between different software versions or modules.

For performance reasons, testing proves difficult, so work is often carried out directly in “Prod.” In the event of an error, live operation suffers.

The configuration is relatively complex and has to be adapted to the hardware. Major PeerTube updates also require changes here.

Conclusion

“I’m also disillusioned about the fediverse generally as a communication medium. A constant battle against spam/LLMs, so much anger, so much focus on follows, likes, and boosts.” - Tom, scitech.video administrator, via email

Tom is frustrated that the attention economy is taking hold here as well. Although conceived as ad-free and rolled out as a small grassroots movement, instances created out of idealism are quickly misused for advertising and self-promotion. The costs also amount to several hundred dollars a year, which further worsens the cost-benefit ratio.

One solution here, according to Tom, could be “a return to Web 1.0 or closed communities,” as already exists on Meatspace, Matrix, Discord, etc.

All the other factors also make it difficult to continue operating his instance: The user base never grew particularly large, and there are only a few people who still upload their content there. As a father, he now has much less time available and must prioritize projects. Thus, he is now ending his six-year journey as a provider on the Fediverse.

I wish Tom continued strength, idealism, and enthusiasm for the future.

Jan’s story: makertube.net

Since I now needed to look for a new hosting platform for my videos, I tried searching for “video-maker” on the PeerTube website. Since my content has a clear tech focus and ranges from research and product development to pre-production, it wasn’t difficult to quickly find suitable candidates.

That’s how I came to my channel on MakerTube.

After successfully registering, I wrote an email to the admins with similar questions to the ones I had already asked Tom.

Motivation - why MakerTube?

“I have a small YouTube channel, and the constant advertising and focus on the big channels, clickbait, etc., and how people are forced to adapt to the content machine, was getting on my nerves. […]” - Jan, MakerTube administrator, via email

Jan was already active on the Fediverse and also works professionally with web services that need to scale rapidly. Therefore, it seemed logical to him to set up his own instance – but of course, with the really interesting topics of this world. It’s not intended as a competitor to YouTube and therefore focuses on makers, musicians, artists, etc.

Is it easy to get started?

Jan mentions it takes a lot of effort to set up such an instance and keep it performing well. Framasoft, who develop the PeerTube software, have done a fantastic job and support the hosting community wherever they can.

I can personally confirm that PeerTube feels completely different in its newer versions than it did a few years ago, and with practical features like automatically generated subtitles, it’s getting closer and closer to the functionality of YouTube.

Jan says that the Fediverse remains a niche. Therefore, the growth of his instance is rather moderate and very organic. PeerTube is currently still something for enthusiastic video creators; the content largely goes unnoticed by the general public.

How much time does it take?

Jan focused on low running costs while maintaining scalability in case the project suddenly becomes large. Now that the system is running smoothly, the effort is a few hours per week for user, content, and support moderation.

Exceptions: Major updates, upgrades, and technical issues.

What are people saying?

“There’s positive feedback, and people are grateful that I run the site and put in the work, but I’d say not very much.” - Jan, MakerTube administrator, via email

But of course, there’s also occasional outrage that this isn’t YouTube and not everyone can immediately create an account.

How do you detect copyright infringements?

“Regarding copyright, there are no direct tools. There’s a report function on the videos to point out violations of copyright or instance rules. Users also use this.” - Jan, MakerTube administrator, via email

Responsibility and liability for small providers like MakerTube lie primarily with the user. However, if violations are reported, the admins must take action and delete content. On MakerTube, however, this isn’t a problem in everyday life: New user registrations are carefully reviewed, and in cases of doubt, the content is inquired about. Most violations concern background music in videos, so there seems to be less awareness of copyright than with uploads of video copies.

And how is it all financed?

MakerTube is financed through donations. According to Jan, especially in the beginning, a lot of time was spent explaining this to people and securing sufficient budget for smooth operation. A video platform is resource-intensive: lots of CPU power for transcoding and streaming, a ton of disk space for all the assets, and simultaneously high bandwidth requirements, especially during peak times.

Conclusion

I’m satisfied with MakerTube so far: Transcoding is fast, videos run smoothly. Even Heise-Verlag has now uploaded their first video to the instance. However, it’s much more difficult to get to the landing page without a “fan base,” which was never a problem for me with scitech.video 😅.

Let me close with another quote from the site itself:

“MakerTube is here to stay.” - MakerTube, March 2025, about MakerTube.

Thanks, Jan. Here’s to the future!

Gitea: stop search indexer

Schallbert — Wed, 12 Mar 2025

After ages, I took a trip to the Google Search Console just for fun. Normally, I’m not interested in it because I don’t use Google myself and assumed that the search engines would do their job well out of self-interest.

I was confused when I realized that the majority of my pages didn’t even make it through the indexer.

The problem

How are people supposed to find my website? Even if they knew the correct search terms, they would hardly see anything on Google. After all, the console clearly stated that none of my blog posts appeared in the indexer. On my Gitea instance, on the other hand, several thousand pages were listed. A clear disproportion. And why does Gitea generate such a high volume of files?

Google’s crawler (a program that searches the web for content and makes it indexable for search) apparently finds every commit, no matter how small, in public repositories as well as the files behind them. Automation runs and other metadata are also indexed.

This is of course completely unnecessary and wastes energy in an area that I would much rather have in the presentation of my blog. So a solution is needed.

Possible solutions

A few possibilities immediately occurred to me:

Configure the crawler for each repository so that only the top folder level is searched.
Completely switch off search indexing for git.schallbert.de.
Use Search Console to make corrections until the indexer shows correct assignments on my blog posts.

The most obvious solution for me was to completely switch off the indexer for my Gitea instance. Correction loops only take effect days or weeks later with Google’s crawler and turned out too time-consuming. I deemed a separate robots.txt for each repository too complicated.

My implementation

I read the Gitea documentation. The suggestion of a robots.txt made sense to me immediately, as did its contents:

User-agent: *
Disallow: /

This tells crawlers from any party * that nothing should be indexed from the root directory / onwards.

Unfortunately, I had no idea how Gitea makes this file available in its instance:

To make Gitea serve a custom robots.txt (default: empty 404) for top level installations, create a file with path public/robots.txt in the custom folder or CustomPath.

Gitea’s configuration directory

So I experimented a bit. I created a public folder with the corresponding file, then a custom folder at different levels within the Gitea folder structure. Each time I restarted the container and checked whether the robots file also appeared on the server.

Nothing.

But then I got rid of the folder names and simply put the file in the Gitea configuration directory - the lowest level at which gitea no longer appears as a folder name. And this was the solution.

Test

After restarting, I was able to successfully display the file in the browser.

In the meantime, Google is also starting to remove the unwanted pages from the indexer. My blog articles will hopefully also be available in Google search over the next few weeks.

Halftones on Dibond

Schallbert — Tue, 25 Feb 2025

Lately, I rediscovered Dibond (aluminium sandwich panel) as a material for machining. I use it less and less for environmental reasons - composite materials are not something I consider particularly sustainable or recyclable. Nevertheless, the beautiful surfaces always have a certain attraction for me. Now then - let’s get down to business!

Blog logo as a point cloud

I was now able to throw an image created by Stipplegen onto the milling machine. To do this I had to solve a problem that was still present in the previous post Milling grayscale images with the CNC: In my most frequently used CAM tool, milling paths for point-based halftone images can hardly be created because each of the over 2200 points would have to be individually assigned a milling operation.

CADasCAM, on the other hand, allows you to add entire groups of elements to one and the same milling action. Now I asked myself which milling cutter I should best work with in Dibond, since I only have a very thin covering layer available.

Simulation radius cutter

I decided on a radius cutter with a half-round head. However, the CAM has no calculation rule for this, so it cannot be selected without further ado. After consulting the manufacturer, the idea came up to simulate the radius cutter for the CAM using a V-cut with a flattened tip, but actually using the ball head on the milling machine.

In order to get suitable values here, I made a few drawings. The table below contains approximate values for the case where the immersion depth does not exceed 0.3mm. I was now able to carry out the path calculation in the CAM and even adhere to a depth limit of 0.22mm. The flattest areas have an engraving depth of just 0.10mm.

When calculating the path, it is important to note that the retraction heights should be as low as possible. Then the Z-axis of the machines does not have to move up so far for every point. I consider a value of 0.5mm to be appropriate.

Radius milling cutter	Simulation by V-Cut
r=3mm	140°, 1.0mm flattening
r=2mm	140°, 0.5mm flattening

Milling process

The motif size for the engraving is 180x180mm and I mill with the following values: Radius cutter 2s r3 6mm, S20000, F1800. I pierce the protective film of the Dibond plate, the thickness of which I do not compensate for. I use a small amount of lubricant to improve quality of cut. After just 11 minutes, the milling process is finished.

Observations

The motif is much clearer in this attempt. Unfortunately, I still have a clear problem with the surface flatness. For example, the engraving in the middle area on the right is too deep (built-up edge - see video - that’s where the milling machine is at the end), while the engraving depth at the very top right and top center does not seem sufficient.

The milling machine and milling cutter have no problem with the material at all. The engraving process sounds a bit like working with a box column drill, interrupted by positioning noises from the stepper motors.
The type of aluminum AlMg1 used for Dibond is soft and chewy. It is not well suitable for machining.
Milling cutters tend to create built-up edges, which increase cutter radius and worsen the cutting quality.
The protective film prevents tears and chip edges.
Applying a little lubricant is a good idea.
Even on a vacuum table, every speck of dust under the workpiece ruins the milling work. A few hundredths of a millimeter difference in height has a major impact on the point size and leads to distortions.
The machine bed must be absolutely flat or the machine must have high-quality, active height compensation; e.g. by means of prior surface measurement using an edge finder. My control software even supports the procedure under the name ZheightComp.

Conclusion

I really like Dibond as a material, but it doesn’t seem to return my affection.
With my technical means, I can’t produce engravings that are repeatable and of good quality.
Maybe I should use plastic or wood instead. Or use a laser…

Finally, a short time-lapse video of the milling process.

Halftone on CNC - Part2↗

Saturn as ASCII art

Halftone images can of course also be created with patterns instead of lines, waves and dots. As an example, I prepared the image of the planet Saturn from Christopher Johnson’s ASCII art for the CNC machine.

CAD/CAM

To get the aspect ratio correct, a font is required in which each character has the same type width. I choose the TrueType font Courier New and adjust the font size and line spacing so that the motif fits on my material and the engraving does not perforate the cover layer.

This forces me to use a 1/8" engraving bit with a 36° tip angle and 0.1mm flattening. I would have preferred a radius cutter here: In soft Dibond, acute-angled endmills cause material to bulge and smear. This leads to unclean lines and a rough surface. However, since the workpiece is very small, the font would be much too wide.

The path calculation shows that I have to choose bold letters on the workpiece, which is only 80x80mm in size, in order to even reach my minimum depth of 0.08mm.

Production

I made the image of Saturn twice: on 80x80mm to test the maximum resolution in the material and on 180x180mm (image below) to check contrast and wider letters.

I am happy with the result. The contrast is high and the silver-white surface of the aluminum (oxide) looks really classy on the matt black paint.

Halftone on CNC - Part3↗

QR codes

While I’m already working with Dibond: I’ll try cutting a few QR codes with a radius bit. This time I’m using different machines.

On the Hobbyline

I engrave in black on the Hobbyline provided to me in overclocked engraving mode. I fix the workpiece with a few dots of double-sided tape. I mill dry. Observations:

The engraving depth seems to increase with increasing line length (built-up cutting edge, milling cutter smears into the material)
In some cases the paint surface at the milling edge bulges upwards
Very dirty milling channels, strong direction dependency

On the Basicline

Here I naturally use my vacuum table and benefit from the much more stable structure. I use a small amount of lubricant. I was curious to see whether the white material still allows sufficient contrast. Observations:

My QR-codeengrave software must be operated correctly. Otherwise, zero points and size ratios shift, as here. Result: My cell phone does not recognize the QR code 😐
The much better hold down prevents major differences in milling channel width.
Nevertheless, you can see the machine’s exit points and sometimes also dragged chips.
The milling quality is still not satisfactory.

Conclusion

In the future, I will have to treat Dibond in a similar way to acrylic glass: use very sharp endmills that cut the material properly. Ideally, single-flute cutters with a hawk’s beak and corner bevel (soft edges), center-cutting, and a decent twist. E.g. this one (link to vhf). This - I believe - prevents material displacement into the even softer core material and also moves chips upwards in the chip channel so that they cannot stick or get under the milling cutter a second time.

I should also work with rapid feed rates. Then the material does not heat up too much. Using a little lubricant and applying protective film contributes to better milling results. Further processing instructions can be found, for example, in the CNC-aus-Holz forum. It may be worth browsing.

CNC-made greyscale images

Schallbert — Mon, 27 Jan 2025

Up until now, I have always created images as engravings in solid colors. This allows texts like this one, but also drawings and real images to be produced with high contrast and attractively.

The goal

With solid color representation, many details cannot be reproduced. But now I would like to bring a photo into the material. Ideally, this should happen quickly, without manual rework and with high image quality.

My CAM tool provides a “halftone image” function for this, which I will experiment with below. Here, too, it is unlikely to get good results of the machine straight away. In addition, the creation of such images is very time-consuming.

Therefore, at the end of the article I will give a few tips and tricks that can help you achieve good results more quickly.

A little color knowledge: Rasterization

The problem of wanting to represent an entire color space with only a few available colors is hundreds of years old. While color gradients can be excellently drawn by mixing colors in the analog world and also on the screen using different luminance values of the three color channels in pixels, not every application has a large number of colors available.

Printers, for example¹. They usually only have the three basic colors cyan (C) - a shade of turquoise, magenta (M) - a shade of pink, yellow (Y) - yellow and key (K) - usually black - at their disposal. From these they then have to create the illusion of all possible colors, because they cannot mix the pigments due to their high opacity. Otherwise the last color applied would dominate or the color would simply run plain. This results in the need to use dot grid or other rasterization processes.

Halftones (halftone image)

One of these raster processes is called halftone. This involves using colored dots or patterns of different line thicknesses to simulate a color tone, which the eye can no longer resolve from a certain distance and therefore “averages” together with the surrounding free space and possibly other colored dots to form a tone value.

This process is suitable for production using a CNC milling machine, provided that the surface of the workpiece to be processed has a different color than the core material.

There are many different production processes for halftones, in which the arrangement of the dots, lines, pattern shape, etc. can sometimes differ greatly. On the CNC, the engraving material used, the required resolution and the desired tool determine the process, since good quality cannot be achieved with every combination.

Dithering

Dithering is more commonly used for display on screens. Unlike with the halftone process, the strength or size of colored dots or patterns does not vary, but rather the spatial distribution of differently colored dots that are sprinkled into the base color. This creates the impression of a mixed color.

Here, too, there are various generation methods: diffusion, noise, patterns, Floyd-Steinberg algorithm…

Material selection

I’ve tried out a lot here. The best materials are those with a homogeneous composition and a high contrast between the surface and core color. In addition, the layer thickness of the surface material should not be too large, as it has to be completely pierced and very large individual dots are then quickly required. The surface is ideally matt. Silk, metallic or high gloss surfaces create a viewing angle dependency that affects the recognizability of the motif.

In addition, the material should be completely flat to the milling surface, as otherwise the dot size varies undesirably.

Suitable materials 👍

Core material	Coating	Special feature
Medium-density fiberboard	Primer + paint	Do not overlap the milling path
Aluminum composite material	Paint or film	Mill to a maximum depth of 0.18mm
Acrylic glass (PMMA)	Paint or film	Use XT (low thickness tolerance)

The least effort for me is processing finished painted aluminum composite material. Practically no preparatory or post processing is required here. However, it is important to limit the milling depth to 0.18mm. Otherwise the PU core will be deformed or tear-outs will occur.

Applying a film can be a good solution if you don’t want to spend time applying paint. Unfortunately, I’m not particularly skilled at working with film. I’m constantly creating air bubbles or inclusions of tiny dust particles. After the milling process, the resulting defects in the height profile are clearly visible in the form of holes in the material that are too large. Plus, when cutting through those bubbles, there’s no adhesive force between film and surface material which can lead to film rip-offs.

Unsuitable materials 👎

In my opinion, the materials listed here are not well suitable for creating halftone images. I have uploaded a few pictures of unsuccessful attempts here. Often, one reason for failure is that the image resolution is too high for the material.

Core material	Coating	Base
Medium density fiberboard	Veneer	Layer thickness too high
Medium density fiberboard	Metal film	Paper core of the metal film is white
Aluminum composite material	Metal, brushed	Material “mushy”, poor resolution
Aluminum composite material	None, PU core	Aluminum pressed into core, poor resolution
Soft aluminum alloys	Paint or film	Aluminum smears, poor resolution

Here I’m experimenting with the pixel density. In the lower area I arranged them too closely so that the coating comes off during milling. On my engraving CNC, such an image on a plate measuring 15x10cm takes about 15 minutes despite overclocking. The material is not well suited for such high resolutions because of the metallic shine, the risk of tearing, hardly achievable surface adhesion, and the high layer thickness.

Halftone images: Requirements for the CNC

For once, the problem here is not related to holding forces, milling forces or difficulties in holding down. On the contrary, when milling images, low forces are applied and that mostly in Z-direction where the machines have highest stability reserve.

The complexity of the jobs is the problem. The portal has to cover long distances and constantly brake and accelerate along the way. This puts a lot of strain on the power drivers, stepper motors and bearings of the linear spindles.

Waves/strokes/lines

In the segment process, the machine moves across an image line by line. Brightness information is usually transferred via milling depth, so that particularly dark areas produce wider lines when using an engraving cutter. This process is ideal for milling in foam, MDF and acrylic glass. It can also be carried out quite quickly because the machine remains in contact with the material and positioning processes are only required for line jumps.

Dot process

Here the size of dots is changed so that sometimes larger and sometimes smaller stipples are created in the material.

Halftone images using a dot process require a relatively long time on the machine. What can be done in half an hour with a hobby laser easily takes two hours with a CNC machine of the same size and resolution (here too there are optimization options).

To produce such images, the machine has to move the Z axis extremely often. When milling points, it has to switch between safety height and material engagement once for each “pixel”.

Example: If 400x400 points are milled (that would be a pretty poorly resolved image at pixel scale), the machine has to set its Z axis in motion 32000 times!

Therefore, the safety height should only be a few tenths above the workpiece surface, e.g. 0.5mm. To protect the heavy Y axis, it makes sense to process the image line by line on the X axis using CAM. This way, the Y axis is only moved for line jumps.

A light gantry milling machine with little mass on the Z axis is advantageous here. It should be well lubricated and the axis guides cleaned. At the same time, it doesn’t need to be super stiff as not to oscillate due to the many abrupt movements.

For stepper motors and drivers, these jobs are marathon races. Their temperatures should always be kept under close observation. I recommend not running the machine with halftone images in continuous operation. The axis and portal loads, on the other hand, are negligible: the machine is hardly exposed to any load in the X and Y directions, as it is mostly in contact with the material downwards and only minimally to create conical indentations.

Labyrinth/line segments

If the entire image is to be covered in just one go, a “travelling salesman problem” (TSP) must first be solved, such as EvilMadScientist’s software Stipplegen2 is capable of. In this process, the density of line segments per area is increased if an area is to appear particularly dark. Such images can be completed quickly on the CNC machine, as they only allow the Z axis to dip once.

The disadvantages of this process are the low achievable resolution (a lot of points are required) and the low contrast, because all lines are the same thickness.

CNC tools

The creation of halftone images is engraving work. It is therefore recommended to use gravers with a 60° or 90° tip angle and an engraving width of 0-0.5mm.

If the motif is small, an engraving width of 0 may well be chosen. The tip angle should be large if you do not want to or cannot mill deeply or if the “light color” is the core color of the material. This avoids deep shadows that darken the image.

If you are machining painted aluminum composite panels, I can recommend half-round cutters with r<2mm and a very low depth of cut at or below 0.18mm. However, this assumes that the machine bed is face-milled and therefore very evenly aligned.

Manufacturing

Image: Attempting to engrave my blog logo as a TSP path with 9600 edges on black painted aluminium composite material. I used a 1/8" 36° graver, 0.1mm engrave width. At the bottom of the image you can see that the material is a little convex and can therefore no longer be reached by the graver.

Image preparation

The image should have a high resolution and be in greyscale. Ideally it should have high contrast and use the entire greyscale spectrum.

The image is now loaded either into a CAM tool that supports the creation of halftone images - e.g. from Vectric, Estlcam - or into an editing tool that can create point clouds or paths, e.g. Stipplegen2 or Inkscape.

There I always try out a few different settings depending on the material used, desired cutter and image effect. You can often see there whether the image production would work and you can assess whether the resolution is suitable for the material used.

How to get good results

Prefer large dimensions: The larger the image is chosen, the less noticeable material inhomogeneities and torn-out spots are.
Allow a large observation distance: The further the image is positioned from the viewer, the sharper the resolution is perceived. Before it eventually appears too small 😆
Take the angle of incidence of light into account: If the material is half-matte, metallic or even high-gloss, the image quality depends very much on the viewing angle and the distance to the light source. The shinier the material, the smaller the area in which the image can be clearly seen.
Use homogeneous, non-smearing materials. I have achieved the best results with acrylic glass.

CNC run

There is one thing you need to bring with you here: time. Below are a few video clips from my experiments (time lapse).

Halftone on CNC - Part1↗

Exceptions prove the rule. Of course there are versions of both screens and printers that get around this problem: They use subpixels with additional colors and can thus cover larger color spaces or - as is common in gravure printing (magazines, packaging, etc.) - use a higher number of colors or color mixtures whose tonal values are closer to the required halftones. ↩︎

Multicolor engravings

Schallbert — Mon, 30 Dec 2024

The project idea

A few weeks ago I was out in the forest. There I saw a beautiful, old sign with the inscription “Help prevent wildfires”. A drawing style like ink from olden times. I took a photo without further ado.

Unfortunately, the sign is quite faded and looks pretty flat as a photo. So I decided to make a three-dimensional copy of the sign with the help of my CNC engraving machine, which would take a little time but not cost much.

From `.jpg` to `.svg`

Before I can create a job for the CNC, I have to prepare the image and put it into vector form. My CAM program can’t do much with pixel graphics. I would like to create the image as a “carve”, i.e. use a tapered engraving cutter to carve the areas of the image that should later appear in color. For this I also need a color separation, as the image should later have white, red and black areas in the milled version.

Image preparation

For this I use common image editing programs such as gimp and paint.net. I use the curves tool to increase contrast, straighten the perspective using 3D rotation and then use the crop tool to get the desired image section. I then erase image defects, dirt or artifacts caused by increasing the contrast. Finally, I add a little sharpness and do a tone separation so that I get pure colors rather than red tones or greyscale.

Now I export the image as .jpg and import it into Inkscape.

Path conversion

Here I first scale the image so that it fits on an A4 page. I also create a small 1mm x 1mm square at the bottom left of the image, which will later serve as the zero point for the two color layer images in the CAM, but will not be milled itself.

As already described in an earlier article on path conversion, I use the trace function and can isolate the red and black areas in separate layers using different brightness thresholds. The “red” layer initially also contains the black image areas. However, I can subtract layers from each other so that after this action only the red parts of the image remain.

I then simplify the paths and remove any path artifacts that were created incorrectly. I export the “red” and “black” layers as .svg files.

Path planning in CAM

In CAM I have to plan two jobs - one for each color. The unprocessed image areas should appear white. The engraving of the first job will be laid out with red color, and the second engraving should later appear deep black.

So I first import the “red” SVG file, select a scaling appropriate for DIN A4, set the zero point at the bottom left and then start creating the carve paths.

Job1: “red” areas

I set a depth limit of 2mm and select a 6mm 90° graver with one cutting edge and 0.5mm flattening at the tip for the engraving. For clearing larger engraved areas, I select a 3mm two-cutter for wood and mill with 60% lateral feed. I set the speed for both milling cutters to 24000rpm and the feed to 3000mm/min.

Job2: “black” areas

For the black areas, I allow a higher depth of 2.8mm for Job2 and save myself the clearing cutter as fewer connected areas have to be removed here. The greater depth should make the black even darker and the motif more three-dimensional.

Fabrication

For the project, I use a black medium density fiber board (MDF). I paint it several times with white acrylic paint until I have a homogeneous and well-covering surface. After everything has dried, I apply a self-adhesive film. Professionals use stencil films (masking films), e.g. from Orafol, but I make do with a much simpler wrapping film for books. Not perfect, as its adhesion is not sufficient for small areas - but good enough for me.

Red flames

Now I run the first job on the milling machine. To do this, I use double-sided tape to attach the workpiece to the milling machine’s working area and align it properly so that the zero point can be set easily.

As soon as the milling job was complete, I put screws in the T-slots of the base plate on the edges of the workpiece. This allows me to position it with repeatable accuracy. Then I remove it from the machine bed, clean it with a brush and apply red spray paint. Unfortunately, this immediately soaks into the MDF (see video below). The flames therefore remain black. I try again, again without success. So I have to go back to the shop and buy a much more viscous paint to apply with a brush.

If MDF is to be painted, spray paint is not suitable because of its low viscosity: it soaks in and leaves hardly any pigment on the surface.

After waiting again until the flames have dried, I mount the workpiece on the machine bed again.

Black firs

Now I run the second job through. This is significantly faster despite not using the clearing cutter, as there is no need to change tools. I then remove the protective film from the workpiece and clean it thoroughly.

How beautiful the sign turned out!

Video of the 3D engraving

There is another time-lapse video for this project, provided on my PeerTube channel.

multicolorEngrave↗

What could be improved

Looking back, there are often things in my projects that can be done better. When I look at the video, two things stand out to me:

1. Path quality determines milling quality

I wanted to get to the result as quickly as possible and therefore did not remove all path artifacts when converting the pixel image to vector graphics. These appear on the finished product as small islands and points that were milled with less depth than the rest. This is best seen in the red areas of the image. In reality, this is hardly noticeable. In the video, however, you can see that the milling machine needs additional time to work out these areas.

2. CAM strategy costs time

My program for milling planning runs milling paths for clearing, outlines and chiseling from top to bottom, line by line. This means that an incredible number of positioning commands are required for non-connected paths. In the worst case each point is approached three times.

G00 commands are bad here because they take a lot of time and cause the machine to move unnecessarily. This problem can only be solved with a more advanced CAM tool.

Repo-Lookout: Fix repository leaks

Schallbert — Fri, 20 Dec 2024

Repo information publicly available

One fine day I got an email from Repo Lookout. It said that one of my repositories was open to access from the internet. This posed a potential security risk as it might contain secret source files, hidden functions or even passwords.

At first I thought this “your repo is not secure” warning was a phishing attempt. But by simply entering the links contained therein, it became clear that Repo Lookout was right and that my Lectures repo was not only public as I had wanted, but that the version control metadata was also openly available on the web server.

Is that bad?

Normally, the internal structure and configuration (actions, discussions, wiki, etc.) behind a repository should not be public. Especially not if the repository is set up as private. But even with public repos, no one should be able to access the structure behind them.

Hence the mission of Repo-Lookout:

“Find source code repositories that have been inadvertently exposed to the public and report them to the domain’s technical contact.” - Repo Lookout /about (Crissy Field GmbH)

In this case, not critical, but unexpected and unpleasant.

Doesn’t do me any harm, because my web server only has the files there for retrieval and even if they were tampered with, they would have had no effect on my repository. I also had all secrets stored in specially created files as described in this article, so that they no longer appear in the configuration files. The repository is also located separately on the Gitea server instance. Nevertheless, only what I consciously want to make public should be connected to the Internet.

The image section from Gitea shows the same commit as the warning from Repo Lookout.

How it came about

In the deploy pipeline for my subdomain lectures.schallbert.de and landing page schallbert.de I have a direct checkout from the Gitea instance to the web server Caddy. This starts automatically as soon as the main branches receive an update. The runner starts a checkout action, which copies the repository to the corresponding directory on the web server.

Checkout action also copies the `.git` folder

The .git folder is simply set up as well. This is where all the data required by the version control software for state management of the repo is stored. I cannot see the copying process itself on Gitea, because the hidden .git folder does not even appear in the directory there. Understandable, because the entire display on Gitea is based on this folder’s content.

So the unwanted behavior remained under my radar - and according to Repo Lookout I am by no means the only one who has this problem.

Option 1: Fix on the web server

The most obvious solution is to block access to the file on the server side. This costs few resources and is easy to set up.

This forum entry shows how to do it: hide-entire-folder-caddyfile. For the files I want to protect, I add the following entries to the Caddyfile:

# /caddy2/Caddyfile

# [...]
respond /.git/* "Access denied" 403
respond /.gitea/* "Access denied" 403

This tells Caddy that when any file /* in the folder /.git is called, it should respond with the error code 403 “Forbidden”. The wildcard (*) is absolutely necessary, otherwise only the folder itself and not the files it contains will be locked.

To check, I check what happens when I request the Git logs:

# Terminal
curl <lectures.schallbert.de>/.git/logs/HEAD
Access denied

Works as designed.

Option 2: Fix in the checkout action

There is, however, a more elegant solution: ensure beforehand in the pipeline that the folder does not appear on the server at all.

Option 1: Using `sparse-checkout`

sparse-checkout allows you to select folders and files that should be part of the checkout. All other files in the repository remain untouched and do not appear in the branch. This saves time and storage space, especially with large repositories. But of course it only makes sense if you already know in advance that not all files need to be touched.

Negative list for `sparse-checkout`

In my case, I don’t want to copy the folders mentioned above to the server using checkout, but I want to copy everything else. How do I do that? Using negation in no-cone mode.

“The user has explicitly said ‘I want these directories and not those directories.’” - Derrick Stolee, Microsoft, on Github

The Checkout Action Guide states that sparse-checkout is also supported for the action automated by the runner.

Now I program with the help of the Github Actions Cheet Sheet:

# /.gitea/workflows/deploy-lectures.yml

# [...]
steps:
 - name: --- CHECKOUT ---
 uses: actions/checkout@v4
 with:
 path: ./tmp
 sparse-checkout: |
 /*
 !.git
 !.gitea
 sparse-checkout-cone-mode: false
 # [...]

To clarify: The script for publishing to my web server uses the action checkout, subfunction sparse-checkout and includes all files in the folder in the root directory tmp and below except .git and .gitea.

What is `no-cone mode`?

By default, sparse-checkout expects a list of folders to include for checkout. In no-cone mode, a list of patterns is expected instead. All operators that can also be used in the .gitignore to specify files, folders, omissions, etc. are possible here. This allows me to exclude certain folders, but has some significant disadvantages. Due to the much higher complexity of the pattern commands, the associated susceptibility to errors and the significantly more computationally intensive evaluation for larger repositories, the use of the no-cone mode is not recommended and is listed as “deprecated” in the documentation. Nevertheless, the proof is in the pudding!

Test with `sparse-checkout`

Now I upload the action to my Gitea instance and let my runner run it once. Then I log in to the web server and check whether the .git folder was created or not:

lectures# ls -la
[...]
drwxr-xr-x 8 root root 4096 Dec 20 10:41 .git
[...]

Damn, the folder is still there. I look in the logs of the action on my Gitea instance:

[...]
hint: 	git branch -m <name>
Initialized empty Git repository in /workspace/schallbert/lectures/tmp/.git/
[...]
::group::Setting up sparse checkout
[command]/usr/bin/git config core.sparseCheckout true

So it’s not because of sparse-checkout. It’s because of the way checkout works: Obviously, the .git folder is absolutely necessary for setting up the repository properly on my web server. So the only option I have is to delete it automatically after checking out.

Option 2: `rm -rf`

And so I try it by force:

# /.gitea/workflows/deploy-lectures.yml

# [...]
steps:
 - name: --- CHECKOUT ---
 uses: actions/checkout@v4
 with:
 path: ./tmp
 - name: --- REMOVE TEMPORARY FILES ---
 run: |
 rm -rfv ./tmp/.git ./tmp/.gitea
# [...]

And finally the .git folder no longer appears on my web server and my “repo leak” is patched. Thanks again to Repo Lookout!

Conclusion

I had the problem that the hidden .git folder, where the configuration and structure of repositories are stored, was published on my web server unintentionally and without my knowledge.

I have presented two working options for solving the problem here:

Set up an access ban on the web server
Modify the pipeline so that it automatically deletes the .git folder after it has been rolled out.

The second option is a bit more complex to implement, but it gets to the root of the problem instead of just fixing the symptoms. It also corresponds to the first principle of data protection: data minimization takes precedence over protective measures.

“What doesn’t exist cannot be lost” - Schallbert

Logrotate-Recursion

Schallbert — Mon, 18 Nov 2024

What isn’t working?

I actually wanted to use logrotate to clean up my logs daily and thus implement data economy: This way, IP addresses that I need for banning can be deleted uniformly and automatically after a short time. Otherwise, I’m not interested in the server logs at all and they just steal storage space.

With my current configuration, logrotate creates one log file per day, but names it xyz.log.1, xyz.log.1.1, xyz.log.1.1.1 instead of the expected xyz.log.1, xyz.log.2, xyz.log.3. This means that the deletion routine no longer works and the number of logs keeps growing.

And why?

I suspect that my logrotate configuration is faulty. At the moment it looks like this:

## LOGROTATE file named 'blog'
blog/* {
# it's ok if the file doesn't already exist
missingok
# Sets the logs to rotate in intervals
daily
# Tells the system to remove old logs and only keep the most recent rotated logs
rotate 7
# Rotated logs will be compressed and kept on disk if they are 10 MB or less.
size 10M
# compress and delaycompress: These two options are used together and indicate that 
# rotated logs should be compressed (gzip) except for the most recent one.
compress

delaycompress
}

As usual, the first thing I do is consult the manual, which I call up in the console using man logrotate. It says that the file to be rotated can be specified directly (and not just selected using a wildcard *). In addition, a log behavior can also be applied to multiple file paths by placing them in front of the curly bracket, separated by spaces.

The asterisk seems to be causing exactly my problem with the cascading logs: This not only rotates the actual target file access.log, but also touches all the files that have already been rotated again.

The solution

According to the manual, I rewrite the processing instructions for Logrotate so that instead of five separate instructions for the respective logs, I only create two files. These are also simpler and shorter than the original version above.

Example:

# rotate webserver log files.
gitea/access.log
blog/access.log
landing/access.log
lectures/access.log
{
 # Sets the logs to rotate in intervals
 daily
 # Tells the system to remove old logs and only keep the most recent rotated logs
 rotate 7
 # compress and delaycompress: These two options are used together and indicate that 
 # rotated logs should be compressed (gzip) except for the most recent one.
 compress
 delaycompress
}

Configuration update

I can roll this out using my existing server configuration pipeline by simply pushing it into the repository on the server. Now I just have to manually copy the files there to /etc/logrotate.d.

Testing Logrotate

Now it would be nice if I could test whether the changes I made actually work. To do this, I use the logrotate user manual again and type:

logrotate -d -v <LOGROTATE-DESCRIPTOR-FILE>

With the debug and verbose flags activated, logrotate will apply the rotation rule specified in the configuration file and provide feedback, but will not actually make any changes or rotations on the file system. For me, the result of the test looks something like this:

lectures/access.log 
 after 1 days (7 rotations)
empty log files are rotated, old logs are removed
considering log gitea/access.log
 Now: 2024-11-28 17:19
 Last rotated at 2024-11-27 00:00
 log needs rotating
considering log blog/access.log
 Now: 2024-11-28 17:19
 Last rotated at 2024-11-16 00:00
 log needs rotating
considering log landing/access.log
 Now: 2024-11-28 17:19
 Last rotated at 2024-11-07 00:00
 log needs rotating
considering log lectures/access.log
 Now: 2024-11-28 17:19
 Last rotated at 2024-09-21 00:00
 log needs rotating
rotating log gitea/access.log, log->rotateCount is 7
[...]
renaming lectures/access.log.7.gz to lectures/access.log.8.gz (rotatecount 7, logstart 1, i 7), 
renaming lectures/access.log.6.gz to lectures/access.log.7.gz (rotatecount 7, logstart 1, i 6), 
renaming lectures/access.log.5.gz to lectures/access.log.6.gz (rotatecount 7, logstart 1, i 5), 
renaming lectures/access.log.4.gz to lectures/access.log.5.gz (rotatecount 7, logstart 1, i 4), 
renaming lectures/access.log.3.gz to lectures/access.log.4.gz (rotatecount 7, logstart 1, i 3), 
renaming lectures/access.log.2.gz to lectures/access.log.3.gz (rotatecount 7, logstart 1, i 2), 
renaming lectures/access.log.1.gz to lectures/access.log.2.gz (rotatecount 7, logstart 1, i 1), 
renaming lectures/access.log.0.gz to lectures/access.log.1.gz (rotatecount 7, logstart 1, i 0)
[...]

The output shows that all the desired log files are taken into account by logrotate and also that the rotation is carried out correctly on a daily basis.

Learned something new again 😃

Klingenbutler

Schallbert — Mon, 14 Oct 2024

Why Klingenbutler?

After I had solved the problem of the lack of a shelf for solid shampoos in my shower - the result was Seifenbutler - I noticed razors flying around there more and more as a nuisance.

If they lie on the edge of the shower, a puddle forms around them. This creates a lot of limescale stains and makes cleaning more difficult. If you put them on the fittings, they will fall down sooner or later and get damaged. The same applies to the wobbly shelf on the edge of the shower cubicle.

It makes sense to reuse the Seifenbutler’s clamping mechanism. This means that only the “front part” needs to be redesigned and I have a stylish shower tray for razors - the Klingenbutler.

The design

I like the round shape of Seifenbutler and want to design Klingenbutler in a similar way. However, the razors should not be touched too easily by accident. So I decide to attach them as “inside” as possible. To do this, I create a ring with an opening, which in turn has three notches for razors. I design the radii of the notches so that all the wet razors I have available fit in there and even toothbrushes can be hung on the Klingenbutler without the bristles touching.

I design the opening (in the drawing on the left) for upper and lower shells so that the upper shell always covers the lower shell in all possible closed positions. This avoids protruding edges with particularly large or small shower rod diameters.

Snapping the razors into place

It was more difficult to plan snap-in aids so that the razors cannot come loose on their own. In the first draft, I decided on a notch in the shape of a double tetrahedron. In the picture above, the notch is labeled “drain”. It has a gradient of 3mm over a length of approx. 15mm, allows splash water to run off and enables the razors to dry quickly.

G2 curves

When I showed one of my designs to a friend of mine shortly before starting production - he is a 3D designer by profession - he immediately noticed that I had not planned for a smooth transition between curve radii. He said that in such cases “G2 curves” are used in CAD and that the model would have much more harmonious transitions.

In technical jargon, this seems to be referred to as geometric continuity. I immediately asked the software company behind my CAD tool whether they supported something like this and lo and behold - there is the tool Create tangent continuous connection. From now on I will use this tool when I connect radii.

The tool creates a Bezier curve whose endpoints are tangentially connected to the two circle segments and thus eliminate the kink.

In addition to the more pleasing appearance, this approach also has an advantage in production: When traveling along bends, the CNC machine has to make sudden changes in direction, similar to connected line segments that meet at an angle. This means that it has to slow down and accelerate at the new curve radius. By using tangent-continuous connections, however, acceleration peaks are significantly reduced and the machine runs at a more consistent speed.

If you have projects with lots of “bends” in the drawing, like with the Klingenbutler, you can quickly add up to a few seconds of production time per part, which can easily be saved - perhaps more on this later.

First prototype - version 1.0

I started production with my “classic” prototype material: cast PMMA (acrylic glass) with a thickness of 5mm in opaque white. If you use reasonably suitable parameters, the material behaves extremely well. At the same time, it mercilessly reveals all errors: here a bevel a little too wide, there slight chatter marks caused by the machine accelerating out of a corner, chips caught in the bottom of threads - all of this is clearly evident in the slightly translucent material composition and strong reflections on the material edges. This material is therefore ideally suited to improving the design and production of products.

Thanks to my experience with Seifenbutler, holding it down during production and assembly of Klingenbutler went smoothly. From now on, toothbrushes and razors can be found in a fixed place in our shower 🙂.

Version 1.1 - Production in bamboo

I actually want to get away from plastic. Over the course of the year I have already switched the production of my Seifenbutlers to “greencast” acrylic glass - i.e. PMMA, which itself is made from 100% recycled material - but it would be even better not to use any plastic at all.

For this I had a few samples of “Moso bamboo” sent to me: stick glued wood cross-glued in three layers, similar to plywood, just under 7.5mm thick. For production in bamboo, I made the fitting tolerances a little more generous (0.1mm undersize) and simply converted metric threads in the plan into simple holes for wood screws.

Omitting unnecessary things

When assembling the first prototype, it became clear to me that the third bayonet had become superfluous. Together with a few other smaller changes, I developed version 1.1:

Omitting the third screw connection in the locking circle
Locking lugs/drain channel: Continuous engraving without stopping at “keyhole”
Bayonet: Chamfering the insides on the back and thus breaking through the surface

Speed and feed

For full groove milling, I use a DLC-coated (diamond-like-carbon) 4mm two-tooth milling cutter without twist with 12mm cutting length. I operate it at S30000rpm with a feed of F3000mm/min. I go into the full material depth with Z+ 7.5mm and drill holes and material ramps at 20°.

I make bevels using a 90° engraving cutter, three-edged, diameter 10mm. I operate it at F5000mm/min at S15000rpm and a center distance of 2mm, which for a smooth edge with a width of 0.5mm already means that it needs 2.5mm of space “downwards” in the Z direction.

For holes, pockets and small recesses I use a relief-ground 2mm two-cutter fishtail with 15° twist and 20mm cutting length. I operate it at a cautious 1700mm/min at S30000rpm and a depth of only Z+1.5mm. Here, too, I dive into the material at 20°.

Production

Working with the material “bamboo”, which was completely unknown to me until then, was surprisingly easy. Thanks to my new and very sharp cutters, there were no tears to complain about and the material itself also had hardly any defects or damage.

Making of Klingenbutler↗

During processing, quite fine chips were created, which are a bit reminiscent of dust from processing MDF boards. I am not yet sure whether I should reduce the speed or whether this is due to the straight cutting edges of the cutter. When milling full slot, the cutter may have to carry chips with it for several revolutions before it can leave them behind.

In any case, drilling with this cutter and unchanged parameters is not recommended. All of the drill holes have charred edges, which suggests that the speed was far too high. I also drilled in one piece instead of retracting the cutter several times. Let’s use the rule of three to make this obvious: At a feed rate of FZ+2000mm/min, S30000/rpm and Z7.5mm, the cutter makes 112.5 revolutions in the drill hole, so the cutting edges rub senselessly along the wall of the drill hole 225 times - only the center cutting edge at the bottom removes material. In addition, the chips are not ejected but are carried along the whole time.

For the next prototype, I will either make drill holes with retractions and a drastically reduced speed or simply use a smaller cutter.

Result

This is impressive. I took the bevel for edge breaking from the production in acrylic glass. It is very narrow at 0.5mm. Nevertheless, the detail is really clean. The upper and lower shells of Klingenbutler can be smoothly moved against each other and the clamp, my problem child from this video, could also be milled without tearing.

Assembly is problem-free and the wood screws can also be easily screwed into the pre-drilled holes. The bamboo Klingenbutler variant holds on to the shower rail even better than its acrylic glass counterpart - thanks to the rougher surface and greater material thickness.

It remains to be seen whether the material is durable enough in damp rooms. It is still popular for cutting boards in the kitchen, so I guess it will be just fine.

Outlook

The use of bamboo has left me very motivated: a great material. Hard and strong but light, easy to machine, dimensionally stable and hardly any material defects: that’s how you want to work with natural materials!

This prototype is now in our shower for endurance testing. In the meantime, I’ll continue to refine the design.

Version 1.2

I have the following small improvements planned for V1.2:

Drilling: Choose a smaller cutter (at the expense of the cycle time)
Fastening: Mill a cutout so that the knurled wheel no longer protrudes
Remove superfluous recess on the inside of the upper shell
Allow different radii to be merged with G2 curves
Replace locking lugs with a more harmonious, round drainage channel
Align the grain: Manufacture all parts “vertically”
Mill the clamps in the first setup and make them 0.04mm thicker

Here too, the aim is to achieve a good product through continuous improvement.

Broken NVME hard disk?

Schallbert — Tue, 10 Sep 2024

The event

Mid-April 2024:

After a system update, my laptop suddenly wouldn’t boot. The bootloader grub-core reports that it is having problems writing sectors.

error: ../grub-core/efi/efidisk.c:638: failure writing sector 0xXXXXXXXX

When I reboot, the screen remains black after the company logo is displayed with a blinking cursor. If I look behind the mask, loading various services fails:

Your browser does not support the video tag.

Boot errors due to read-only nvme

Initial analysis

I suspected a problem with the computer update and carried out a hard drive test from Lenovo’s UEFI. Result:

The media check is complete, the result is: FAIL

It’s not a problem with the operating system, but rather with the disk. Damn. Do I have to worry about my data? I do have backups lying around, but the last one is three weeks old, so I might be missing several hours of work. A setup from blank will take me a lot of time. So I’m going to try to see if anything can be salvaged.

Self-test

So I run the SMART tests for the disk with smartctl -x /dev/nvm0 and now know that it counts about 7600 hours of operation and has 240GB of usable capacity. The disk used under the name nvme0n1 thinks it is now “read-only”. All partitions including the kernel. Great. The result of my online search was something like:

The disk is irreparably damaged. A new one is needed.

Repair attempt

You don’t have to believe everything you read online. So I use Lenovo’s own repair tool for hard drives. It runs from the UEFI screen and is called Bad Block Recovery Tool. The program scans the disk for defective sectors and attempts to restore affected blocks. The run takes almost half an hour for the last 10%.

Result: Recovery failed. I run the tool again. And a third time in slow pedantic mode. Again and again to no avail. I keep getting critical medium error.

Is all of this not working because the disk has switched to read-only mode?

As I can no longer boot and have no console available, my options with on-board resources are now exhausted.

Comment: The read-only mode for SSDs

According to the Internet (DiskTuna), the read only mode is there to limit damage. If the controller of the storage device detects a serious error, it puts the disk into read-only access. This minimizes the risk of further blocks being corrupted or killed.

In the forum Superuser someone describes an error identical to mine, and there is actually nothing that can be done to keep the medium working.

Data recovery using a boot stick

I continue with a USB stick that I convert to a Linux boot stick at another computer. I plug it into my laptop and can finally boot it up again. I connect an external hard drive as data recovery medium.

Get an overview with fdisk

First I check whether all disks and partitions appear. To do this I type the command sudo fdisk -l into the console.

Luckily, everything is still there! Maybe I can just copy the files from the defective hard drive?

Copying with cp

Let’s try it out.

sudo cp -r -v /mnt/nvme0n1p3/Documents/my/important/files /mnt/externalHdd/save

For a few seconds I think everything is going well. But then cp stops with an error message saying the volume is not available. In fact, the disk is no longer mounted. However, I can find it again and mount it straight away. A few attempts later, I have backed up my really important data, but the copying process always stops after a few seconds. This would take far too long for the entire disk.

The observed behavior - mount, copy for a few seconds, disk disappearing - was confirmed to me in a few related forum posts online, so it seems to be normal behavior.

I need another solution.

Save data with ddrescue

After a quick online search, I installed the program ddrescue.

The documentation states some things very clearly. For example, that you should understand exactly what it does before you use it. Or the following:

“Never try to repair a file system on a drive with I/O errors; you will probably lose even more data. " - ddrescue Manual

Now I’m wondering whether I made the same mistake with the block repair attempt above. Or am I on the safe side with Bad Block Recovery, since this is about data blocks on the disk and not the “overlying” file system?

It also says here that the so-called mapfile is a core function of the program. It offers a great time advantage when aborting, restarting, merging backups and during data recovery. Therefore, it is not a bad idea to create the mapfile when creating an offline backup and to put it away safely. Overall, I found the tool extremely helpful, effective and easy to use and can only recommend it at this point.

Back to action: I find a very simple command to copy the entire data partition of my damaged NVME storage to an external hard drive:

ddrescue --sparse /dev/nvme0n1p3 /run/media/liveuser/myExternalHDD/datapartition

The --sparse option skips areas of the source disk that are filled with zeros and can thus save a lot of space on the target disk.

The program takes a quarter of an hour to complete the entire run. As you can see in the picture above, it can actually save 99.6% of the data and even several months later I haven’t missed the remaining 0.4%. Lucky me.

Verification: View the output of ddrescue

In any case, the program ultimately saves a single file on my external disk: datapartition.img I can view this image with fdisk -l:

datapartition.img: 236.89 GiB, 254356226048 bytes, 496789504 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Everything is there, that’s good. Is all data included as well?

To check this, I mount the partition and look around using cd and ls -l:

sudo mkdir /mnt/olddisk # Mount point of the rescued data image
sudo mount /run/media/liveuser/myExternalHDD/datapartition.img /mnt/olddisk

Fortunately, everything that should be there was there. Now I could unmount the image, shut down the computer and have a new hard drive installed.

The new hard drive

Here, too, I was lucky: my used laptop was still under warranty and so I got it returned within a week with a new (and twice as big) hard drive. Because I was so afraid of data theft, I removed my old hard drive before sending it. I felt pretty paranoid.

I could have saved myself this work, maybe, because ddrescue has a software solution for paranoid people like me: the --fill-mode option. This allows you to delete the “still good” blocks and sectors after the rescue has been completed. Only the sectors that are already defective remain. From the online documentation:

ddrescue --fill-mode=+ --force /dev/zero bad_drive mapfile

I’m not sure whether this would have worked with a disk like mine, which the on-board controller had set to “read-only”. Whatever.

At least I can now continue writing blog entries with a fully functional laptop ☺️.

Gitea act_runner: Jump-start issues

Schallbert — Fri, 30 Aug 2024

Problem statement

Since I run this page myself I experience problems starting Gitea and act_runner. Sometimes, the runner won’t start. It exits with status -1 and so my workflows wouldn’t run the website build, verification and deploy tasks.

The error message is always the same:

[...]
runner-1 | time="2024-08-26T10:09:46Z" level=info msg="Starting runner daemon"
runner-1 | time="2024-08-26T10:09:46Z" level=error msg="fail to invoke Declare" error="unavailable: 502 Bad Gateway"
runner-1 | Error: unavailable: 502 Bad Gateway
runner-1 exited with code 1
gitea | 2024/08/26 10:09:46 cmd/web.go:242:runWeb() [I] Starting Gitea on PID: 16

Ok, the runner cannot hook onto Gitea. When I restart all services via docker restart, the act_runner container tells me it would be missing a network with ID <long hexcode ID>

Interim corrective action

I didn’t find time for resolving this issue. So I just ran docker compose twice:

# schallbert server-console
docker compose -f /path/to/giteas/composefile up -d

The second time, act_runner would start successfully (because Gitea is ready):

gitea | 2024/08/26 10:38:17 routers/init.go:116:InitWebInstalled() [I] Git version: 2.45.2 (home: /data/gitea/home)
runner-1 | time="2024-08-26T10:38:22Z" level=info msg="Starting runner daemon"
runner-1 | time="2024-08-26T10:38:22Z" level=info msg="runner: action-runner, with version: v0.2.10, with labels: [ubuntu-latest], declare successfully"
runner-1 exited with code 0
runner-1 | time="2024-08-26T10:40:32Z" level=info msg="Started runner daemon"

Not very satisfying to do things twice. Plus, I did this manually. So why is that?

Root cause analysis

First I thought it was due to Caddy not being ready as it runs the reverse proxy, connecting Gitea to the internet. In reality, Gitea is the culprit: At the time, docker starts act_runner, its startup procedure wouldn’t be complete. So I try to find out how to manage dependencies in docker.

Solution: Reflecting dependencies

First, I try linking gitea to the runner in the docker compose file.

depends_on

# gitea/docker-compose.yml
 ## service: runner
 ## [...]
 depends_on:
 gitea:
 condition: service_started

Well, still act_runner seems to be starting early. Docker just makes sure it has started the container, and does not wait for it to return a healthy signal.

Healthcheck

So I have to make sure that gitea’s startup procedure is complete. For this, docker provides the condition service_healthy. I adjust the runner configuration like so:

# gitea/docker-compose.yml
 ## service: runner
 ## [...]
 depends_on:
 gitea:
 condition: service_healthy # required so runner can attach to gitea
 restart: true

The service_healthy qualifier is returned by the healthcheck function. It consists of a so-called test and some environment parameters that define intervall, number of retries, delay and timeout conditions for checking service ready.

For the test I chose a simple command, assuming that Gitea (and, implicitly, Caddy too) would be healthy once it is able to respond to a GET request from curl.

# gitea/docker-compose.yml
## service: gitea
## [...]
 healthcheck:
 test: ["CMD", "curl", "-f", "https://git.schallbert.de/"] # checks if gitea is available
 interval: 10s
 retries: 3
 start_period: 30s
 timeout: 10s

With these lines, docker checks if gitea is “healthy” and only then starts act_runner. This permanently solved the problem and both Gitea and act_runner are stable.

Sending error logs

Schallbert — Tue, 20 Aug 2024

In this article I’ll look at how to set up “monitoring” for my server. Applications and services should be able to send me notifications in the event of an error.

What is this about?

Learn about and select sending mechanisms
Write test messages and verify the automation
Automatically send error report from Borgmatic
Send runner logs through Gitea
Notification when logging into my server via ssh
Create and send logs for server updates / server errors

Verification and monitoring

After running an automation, I want to know whether it was carried out successfully and whether all programs and services started their work as expected. This should apply to any automation - whether it is specifically sending an update, creating automatic backups or an action from Gitea, it doesn’t matter.

Normally I would use reporting mechanisms from act_runner for something like this. In the case of a server update, the runner is not available due to the circular reference already mentioned in the article Rolling out the server configuration, as all applications have to be shut down temporarily.

In addition, applications may have their own procedures for monitoring. So I have to look at mechanisms that allow me to easily access the information.

What characterizes good monitoring for me?

It is unobtrusive, so it only reports in the event of an error or unusual occurrence.
It provides specific information and error messages that are easy to understand.
It uses a message channel that works even if the system being monitored crashes.
It presents reports and error messages in isolation from other topics and does not mix anything.
It is brief.

Methods of automatic reporting

I will break this section down. In the general part, I will discuss the on-board tool for asynchronous monitoring that is available to me on the Ubuntu server. After that, I will look at the solutions that are partly built into my services or the applications that are compatible with them. I do not want to limit myself to the classic tool of email, but also look at “more modern” communication channels such as messenger programs or RSS feeds.

Mail via console - curl

Sending emails as notifications is common practice in many companies. With Linux, this can usually be done without any additional programs: The standard curl can help here.

curl is a program for transferring data from or to a server. If I enter my blog as the target, I get the HTML page output as a text file on the console:

curl https://blog.schallbert.de

This made me notice how much unnecessary data my blog software generates. I’ll have to clean that up later. Back to the topic: You can also use curl to access any web backend - for example a mail server:

# copied from https://stackoverflow.com/questions/8260858/how-to-send-email-from-terminal
curl --url 'smtps://smtp.gmail.com:465' --ssl-reqd \
 --mail-from 'from-email@gmail.com' \
 --mail-rcpt 'to-email@gmail.com' \
 --user 'from-email@gmail.com:YourPassword' \
 -T <(echo -e 'From: from-email@gmail.com\nTo: to-email@gmail.com\nSubject: Curl Test\n\nHello')

This only works for my mail provider if I allow logins from external clients. Google, for example, calls these “less secure apps”. As described in my post Server configuration with Git, I’m not a fan of writing secrets into anything - so I would rather not use the direct route via curl.

Mail via console - mail, mailx, mailutils, swaks

If you don’t want to always provide all the configuration information for the server and secrets, there are various handy tools for the console such as mailutils or swaks. Here, the connection to the mail server is configured once using the tool and can be stored in environment variables, for example. The syntax varies from program to program, but an email always drops out at the end.

The only major disadvantage for me is that information domains are mixed up. I would not want another “report thread” in my emails that I’d have to search for in the mass of messages. Other notification media in contrast allow me to set an automatic expiration date so that they disappear from my list after a set time.

Create RSS feed

Unusual but possible: I could create the monitoring as an RSS feed like my blog (schallberts-blog-feed) has e.g. using a Jekyll instance and put it online as a website. This would be easy to subscribe to, were readable with practically any reader and I could even set it up separately for each application. But there are obvious disadvantages:

High effort: Gitea runner with Jekyll instance, web server and subdomain required.
Publicly available: Suddenly build processes, updates, upgrades and error messages are accessible to everyone.
Error-prone: If Gitea, act_runner, my proxy or the web server crashes, I don’t get any reports.

With the last point at the latest, this option is out of the question for me. I want to get a report when my applications don’t do what they’re supposed to. Let’s take a look at the applications I already run and see how they tackle this problem.

borgmatic

Borgmatic is compatible with a lot of monitoring options. These include apprise, ntfy, healthchecks, cronitor, pagerduty, cronhub and grafana.

Apprise is a library. It is open source and can be integrated into an existing application as a dependency. Like an adapter, it enables asynchronous communication between the application and various communication services such as SMS, mail, messenger (e.g. Signal), various home automation systems or the notification mechanism of various operating systems. The latter, however, only works on the local machine. The trigger for communication must always come from the application.
ntfy is a push notification service. It is open source and can be self-hosted or used as a service via web application. Apprise, for example, supports ntfy as a communication service. The structure is quite simple and works like MQTT via publication subscription / broker client mechanism, but HTTP-based.
Healthchecks is a service. It is open source and can be self-hosted. The service is there to monitor regular activities and can act as a dead man’s switch: If, contrary to expectations, there is no response from the monitored program, it can raise an error message itself. This can in turn be forwarded to various communication services.
Cronitor is a monitoring solution and web application that, in addition to the notifications I need, provides a lot of analysis tools, performance measurements and metrics - mostly for money. There is a free “hacker” account with limited functionality, but this tool is also far too big and complex for me.
PagerDuty sees itself as a commercial “operations” platform that provides “incident management”, automation, “business operations”, “AIOps” etc. It’s out for me straight away. At the latest when I read the word “AIOps” 😅
cronhub looks like a commercial web application to me that, like Healthchecks, can create cron jobs, monitor them and report errors. It is of no interest to me because it does not seem to be open source and I could not host it myself.
Grafana is an open source web application that can either be self-hosted or used as a cloud service. Although many larger companies and projects use the application, it is orders of magnitude too extensive and feature-rich for my purposes.

fail2ban

Fail2ban does not have an automation solution like borgmatic. It simply creates log files that need to be evaluated in order to obtain content for notifications. At the moment I cannot think of anything that I absolutely need to know about Fail2ban. So I am not creating any reports for this for now.

Gitea

Gitea allows the log files to be configured very precisely. Access logs can be written out separately from service logs, repository logs or action logs and then processed further. According to my research, Gitea only offers a mailer as a notification system. In my opinion, this is primarily intended for repository and action logs.

Here, too, it would be most beneficial for me to analyze the logs and create a report myself if necessary.

Server

I already roll out my server configuration files using a script. So, in the event of an error, I could redirect the logs to a file and attach them to a report in any channel.

In addition, successful logins on the server would be worth a message. Then I can immediately determine whether it was me or not.

Selecting the reporting program

I not only have to select a monitoring program, but also choose a communications service through which the reports are sent.

Program	Advantage	Disadvantage
curl	simple	configuration must be provided
	works super easily with ntfy¹	No monitoring if the server crashes
	Onboard tools
	no service required
Apprise	enables integration of the Signal API²	Complex in interaction with external services³
	no service required
Healthchecks	Can act as a watchdog/dead man switch	Registration required
	Can be hosted locally⁴	Local hosting contradicts the watchdog concept

I only have a small server with a few applications, so I keep it as simple as possible and choose Apprise for services like borgmatic, which come with this library anyway, and curl for those whose reporting mechanics I have to program myself.

Communication channel

Program	Advantage	Disadvantage
SMS	High reliability	Registration required
		Limited to a few characters
		No attachments possible
		No topics, domain mixing
Mail	easy to set up	Poor searchability
RSS feed	Good sortability	Complicated to set up, error-prone
	Client very lightweight	Local hosting contradicts watchdog concept
		Content publicly available
ntfy	Simple and lightweight	Registration optional for web use
	Client purpose-oriented	Insecure: No encryption without registration
	Free for small users
	Independent of your own machine
Signal	“Note to self” easy to set up	Setting up and configuring the Signal API complex
	Independent of your own machine	Topics somewhat difficult to implement
		Useless if the server crashes

SMS, mail and RSS feed are already out of the question for me due to some of the disadvantages already explained above. So that leaves ntfy and Signal. ntfy impresses with its simplicity: send an HTTP PUSH request to a self-defined “topic” and subscribe to it on your cell phone - done. It is also easy to expand, because with self-hosting I can later increase security if necessary (encryption) and control the sending behavior. Signal, on the other hand, requires a separate client with relatively complex configuration. In addition, the devices would have to be connected to the server and cell phone, so it is not easy to add new subscribers. On the other hand, the transmission is well secured end-to-end, I do not have to register and it is also free.

For now, I have decided to go for the less complex solution with ntfy.

Logging

Good. Now it’s clear that I’m calling home using curl and apprise via the ntfy service. Let’s see what content needs to be transmitted and how to make it as unobtrusive as possible, but still short and concise.

When should logs be sent?

borgmatic: If a backup fails
If I successfully log in to my server or Gitea
After rolling out an update to the server configuration
Docker: If an application fails or doesn’t start
Gitea: Failed run of act_runner

What should I include in a report?

I want the classic “W questions” answered.

When did it happen (time stamp)?
Which application is reporting?
What happened?
Where (module, line of code) etc. did it happen?
How many were injured (severity, recovery)?

A notification then looks something like this:

TIMESTAMP APPLICATION PRIORITY MESSAGE EFFECT DETAIL

Rough version of a report

An ntfy message could look something like this:

curl \
 -H "Title: Error Borgmatic" \
 -H "Priority: urgent" \
 -H "Tags: warning" \
 -d "YYYY-MM-DD HH:MM:SS Backup creation aborted. Access to repository blocked" \
 ntfy.sh/schallberts-topic

I install the corresponding app on my phone, register on the “Topic” and send the message. It’s nice when things just work! Aha, the app shows the time of receipt. That’s accurate enough for me.

Implementation

Here I’ll take a look at all the services for which I’d like to set up notifications one by one.

borgmatic

Let’s start with a pilot test in small steps. First I configure Borgmatic to send a message via Apprise to ntfy if the backup creation fails:

# /borgmatic.d/config.yml
on_error:
- echo "Error while creating a backup."
- apprise -vv --title "Borgmatic Error"
--body "Could not run {output}. Aborted {error}."
ntfy://schallberts-topic

Then I test the command by calling apprise within the Borgmatic container. And indeed, it works. But getting here took me an hour, as the yml with its syntax rules even interprets within strings and borgmatic constantly refused to read the configuration file due to : and - characters. If you see an error similar to this one:

At 'on_error[1]': {'apprise -vv --title "Borgmatic Error" --body "Could not run {output}': 'Aborted {error}." ntfy://schallberts-topic'} is not of type 'string'

This shows that the interpretation of characters or indentations went wrong and the punctuation needs to be checked. Alternatively, the pipe operator | can be used to combine a command. Reference: yml specification

Server

An application of ntfy to monitor logins on a server can already be found in the documentation of ntfy itself. The description shows that something like this is easy to implement yourself and gives a great example using Pluggable Authentication Modules (PAM library).

Insert the following code at the end of the sshd file in the etc/pam.d directory:

session optional pam_exec.so /usr/bin/ntfy-ssh-login.sh

This line tells PAM that when a session is opened via ssh, it should call the executing module pam_exec, which then runs the script specified below. The value optional means that the configuration file should continue to be run even if the action fails. More details on how to use PAM can be found, for example, on Baeldung.

Then you simply have to store the ntfy call in the ntfy-ssh-login.sh script when the script determines that PAM has detected an open_session event.

That’s exactly how I implemented it and it works straight away. Great! The only disadvantage: I made this modification directly on the server. Without a container and outside of my configuration backup. If I now have to set up the server again for some reason, the change in PAM is lost and I won’t receive any more notifications until I manually enter the change again.

Server configuration

To send a report after running the configuration automation, I actually only have to adapt the server-config-action script that I wrote in the last article.

First, I want to be informed when the script runs without errors. To do this, I add a curl command at the end of the file.

# server-config-action.sh

# action commands...
# [...]
# send success notification
curl \
 -H "Title: server-config-action" \
 -H "Priority: low" \
 -H "Tags: white_check_mark" \
 -d "Rollout successful" \
 ntfy.sh/schallbert-server-config-push-topic

However, if the script does not run without errors, I would like to do the following:

Cancel actions after the error occurs
Create an error log
Send this log

I can achieve the first point by adding a trap for errors:

# server-config-action.sh
trap 'handle_error $LINENO' ERR

# action commands...

This calls the function handle_error. It is given the line number where the error ERR occurred. It also covers errors that can occur when restarting the container. I create the error log by redirecting the output of the individual script commands. I achieve this with the following line:

# server-config-action.sh

exec 3>&1 1>server-config-action.log 2>&1

# action commands ...

With this command, any standard output stdout should be passed on to the file descriptor 3, specified here with the log file server-config-action-log, overwriting it. For “append” there would have to be two redirection operators >>⁵. The log file thus replaces the console output, which I still had at this point in the previous article.

To have the log sent to me, I now define the following function at the beginning of the Bash script:

# server-config-action.sh
# on error, send a notification
handle_error() {
 # stop redirecting to file
 exec 1>&3 1>&2
 curl \
 -H "Title: server-config-action FAILED" \
 -H "Priority: high" \
 -H "Tags: x" \
 -T server-config-action.log \
 -H "Filename: server-config-action.log" \
 ntfy.sh/schallbert-server-config-push-topic
 exit 1
}

# set error trap
# redirect stdout and stderr to file
# action commands...

Gitea

For Github Actions reports, there is already an existing example at ntfy_examples/#github-actions. And the best part: Since the act_runner from Gitea is compatible with Github Actions at in the environment parameters, sending to ntfy works for me straight away. In the runner’s workflow file, all you have to do is enter the curl command specified on the website and you’re done.

Result

Five notifications about the most important processes on my server have now been set up. I have understood the underlying mechanics and can create additional notifications at any time if I need them. If I trigger all notifications as a test, my smartphone will display the as shown.

But I have not yet achieved independence from the system to be monitored. All notifications come from the affected machine and there are sometimes even dependencies between Docker containers. For example, the act_runner has to connect to Gitea via websockets. And that only works if Caddy provides the reverse proxy.

If I find out in the next few months that a service “under my radar” is no longer able to work, I will have to establish independence.

Context for setup with ntfy ↩︎
Context for the Signal API ↩︎
Report by asad-awadia on setting up the Signal API ↩︎
Healthchecks is directly available as a Docker image. ↩︎
Incidentally, the operators << and >> are not difficult for me to remember, because I used to have a lot to do with the programming language C. There - and in many other programming languages too - these are shift operators that can “shift” a value bit by bit or move one field to another. In the RISC architectures that I used at the time, the shift operation was “cheap”, i.e. very fast and memory-saving. ↩︎

Server configuration rollout

Schallbert — Wed, 31 Jul 2024

In the previous article, I brought my configuration files under version control. Now I want to automatically install the updates provided on the server.

What is this about?

Writing a script to automatically detect the update trigger
Server applications should be shut down and a backup copy created
The script should distribute the configuration on the system
All applications should then be restarted

Ways and possibilities

Here, too, I spent several hours researching. For larger projects, infrastructure experts seem to use specialized automation tools. These include Ansible, simpler tools such as cdist or even Kubernetes for highly scalable services.

Automation services Ansible, cdist, Kubernetes

Ansible and cdist seem to follow a similar concept: On the source machine (in this example my laptop) I create the configuration for my server and store this and the instructions for configuring my services in a “playbook” (Ansible) or in “types” (cdist).

To put it simply - as I understand it - the tool then takes care of building the configuration at the push of a button, dialing into the target host via ssh, pushing it over there and then starting it. For Ansible, there are even tutorials like this one for my scenario with docker-compose, which makes getting started even easier.

Kubernetes takes a different approach. It sees itself as more of a container manager, load balancer and scaling agent, but can do similar things for my purposes.

My approach

I, on the other hand, only need a fraction of the capabilities of these programs. I am also put off by the “additional” ssh channel, the configuration effort, the additional programs sometimes required on the target system, and the necessary reading and selection of the best tool for me. Because thanks to my very simple pipeline from the last article, the configuration is already on my server. It “only” needs to be copied to the right places and the affected services restarted.

Therefore, I am trying to solve this problem using on-board tools, my brain in working order and a few searches in relevant forums on the topics “executing a script when a file changes and “copying folder structures in Linux”.

Risk of circular reference

However, I am taking a risk: Since the deployment runs via Gitea, but the configuration affects Gitea itself, if there is an error in this module I can no longer change or reset anything: The Gitea service is then broken. I would have to get the configuration up and running again manually on the server.

I’ll try it out anyway and see if I am actually confronted with such a problem. If so, I’ll just switch to cdist and document it in a separate article! 🤗

Preparation: Create folder system and scripts

I think about it for a moment and create a few folders in the server file system:

# /opt/server-config
mkdir automation-hooks-trigger
mkdir automation-hooks-handler

These folders should represent the two sides of the automation. trigger contains text files that can be manipulated from the (web) service side. For example, act_runner should write the file server-config-update as soon as an update is available.

service –> writes to trigger file ||| server_handler() –> trigger_file.changed ? run_action() : loop()

The handler folder contains script files that make the necessary changes to the server file system.

Implementation: Shell script to handle the update

In the previous article, I executed the command touch server-config-update.txt in the act_runner container to signal the presence of a new server configuration. On the server, I now use the following code to periodically check whether this file has changed.

### set directories
actionfile=/opt/server-config/automation-hooks-handler/server-config-action.sh
triggerfile=/opt/server-config/automation-hooks-trigger/server-config-update.txt

### Set initial time of file
LTIME=`stat -c %Z ${triggerfile}`

while true
do
 ATIME=`stat -c %Z ${triggerfile}`

 if [[ "$ATIME" != "$LTIME" ]]
 then
 ${actionfile} 2>&1
 LTIME=$ATIME
 fi
 sleep 10
done

The operating system is given the interpreter with which the script is to be executed via #!/bin/bash. The command stat -c %Z queries when the file was last changed and returns the time in Epoch format. Finally, the time taken at the beginning of the script is compared with the time taken in the loop and if there is a change, the deploy routine that is still to be written can then run. The reference time is then updated. Finally, the script (sleep 10) pauses for ten seconds before the query starts again. I use absolute paths so I’m able to both start it from console and per service cron oder systemd.

First test and hooking into “autostart”

Now we need to make the file executable for a first test:

# change file mode bits: add "executable" flag to server-config-handler script
chmod +x /opt/server-config/automation-hooks-handler/server-config-handler.sh

Next, I create the server-config-action.sh file and add just an echo command. To test the setup I create my update trigger file locally and start the script. I then modify the triggerfile in another shell using touch server-config-update.txt and --- CONFIG UPDATE TRIGGER detected --- appears in the console. Great!

Later on the server, I need to have the script run automatically after a restart. To do this, I use the cron tool help and create a new entry using crontab -e:

# cron can automatically execute recurring tasks
# in this case, we're running server-config-handler script on reboot
@reboot sh /opt/server-config/automation-hooks-handler/server-config-handler.sh

With the command ps aux I can now check whether the script is actually being executed:

schallbert@server: ps aux
[...]
8:15 0:00 /bin/bash ./server-config-handler.sh
8:15 0:00 sleep 10
[...]

Backup

Making a backup of my applications and files before I roll out the update makes total sense. So I tell Borg that I want to create a backup now. Of course, I have to stop all services first. This means that all data is accessible, coherent and static.

Freeze state and data

To do this, I create a script in automation-hooks-handler that automatically terminates all containers except borg.

#! /bin/bash
# /opt/automation-hooks-handler/backup-pre-action.sh
# this shell script shuts down all docker containers prior to backup

echo "Shutting down containers for backup:"

echo "watchtower..."
cd /opt/watchtower
docker compose down 2>&1
# [...]

The expression 2>&1 means that any error output is redirected to the console. The number 1 represents the file descriptor for stdout, while 2 means stderr. The operator >& functions as a redirect merger. Later, we can go to this point and write the output to a log file - but I’ll leave that out for now for the sake of simplicity.

Borgmatic: Trigger handler mechanism #2 and #3

I can now run the script in two ways:

As a call by the server-config-handler script described above
By the automation solution borgmatic placed in front of borg

I choose the second option and therefore write the following commands in borgmatic.d/config.yml:

# borgmatic.d/config.yml
# List of one or more shell commands or scripts to execute before
# creating a backup, run once per repository.
before_backup:
 - echo "Triggering container shutdown for backup."
 - touch /etc/automation-hooks-trigger/backup-pre.txt
 - sleep 20
 - echo "Assuming container shutdown complete. Creating the backup now."
# [...]
after_backup:
 - echo "Triggering container restart after backup."
 - touch /etc/automation-hooks-trigger/backup-post.txt
 - sleep 10
 - echo "Assuming container restart complete. Exiting."
# [...]

For this to work properly, I have to create a volume in the associated docker-compose.yml and have it point to the path in the server’s file system: ${VOLUME_UPDATE_TRIGGER}:/etc/automation-hooks-trigger Now I set up the other side of these triggers: Handlers monitor trigger files for changes and call action scripts accordingly. These look very identical to server-config-handler.sh except actionfile and triggerfile paths.

Creating the backup

If I were to address borg directly, the backup could be created using create. To do this, I would have to specify in which repository the backup copy should be saved and under which name. In the example below, this is specified using the scope operator: ::config-update. The folders to be backed up are then specified.

borg create /path/to/repo::config-update ~/opt

I use borgmatic, which takes a lot of work off my hands using the configuration file. However, I have to execute the command in the container. To better check whether everything is working, I output statistics “verbose” to the console (--stats -v 1) and display the copied files --files.

docker exec borgmatic sh -c "cd && borgmatic --stats -v 1 --files 2>&1"

I add this line to the automation script.

Second test to create the backup

If everything works now, the complete process looks like this:

After the changed files have been received by Giteas Automation, act_runner (in the Docker container) stores the files on the server and then sets the server-config-update trigger.
Within 10sec the trigger is recognized by server-config-handler.sh, which then calls server-config-action.sh.
borgmatic (in the Docker container) is instructed to create a backup. This in turn writes the pre-backup trigger.
Again within 10sec this pre-backup-handler.sh calls the backup-pre-action.sh script and stops all containers except borgmatic.
Due to the built-in delay, borgmatic waits for this and then creates the backup.
After the backup, borgmatic writes the backup-post-action.sh trigger.
Within another 10 seconds, post-backup-handler.sh recognizes the changed file and restarts all containers via post-backup-actions.sh.
borgmatic tells server-config-update.sh whether any errors have occurred anywhere in the process so far. If not, it continues.
All Docker containers are stopped.
The server configuration is rolled out to the appropriate locations.
All Docker containers are restarted with the new configuration.

A first success: These scripts are already running on my laptop up to point 6:

schallbert@laptop: touch server-config-update.txt
server-config-handler@laptop: --- RUN backup ---
borgmatic@docker: /etc/borgmatic.d/config.yml: Running 4 commands for pre-backup hook
 Triggering container shutdown for backup. 
backup-pre-handler.sh@laptop: --- RUN backup-pre-action.sh ---
 Shutting down containers for backup:
 watchtower...
 [...]
 complete.
borgmatic@docker: Assuming container shutdown complete. Creating the backup now.
 local: Creating archive
 Failed to create/acquire the lock /mnt/repository/lock.exclusive

Troubleshooting for “failed to acquire the lock”

This problem occurs for me when borg reports an error when creating a backup that causes the program to abort. In this case, the repository is apparently not released correctly, so that after restarting the container it remains reserved for the old, now non-existent container. The following command solves this problem:

docker exec borgmatic sh -c "cd && borg break-lock /mnt/repository

Transferring the configuration

Now the configuration files have to be copied to the correct location on the server. Fortunately, I had already cloned the target folder structure when creating the repository, so I should be able to do this with a single copy command without “hardcoding”. After a bit of online research and a look at the user manual for the copy command man cp, I have my command:

# copy recursively contents of folder "server-config" to "/opt"
sudo cp -r -v /opt/server-config/. /opt 2>&1

This tells the operating system to copy the contents (/.) of the folder server-config recursively (-r), including all subfolders into the folder opt, which is located in the root directory /. cp works in an overwrite-supplement manner, so it will create files that do not yet exist and overwrite existing ones, and will not “copy them next to each other” under the same name. With the -v option I can output additional details, and with 2>&1 I redirect the error output to the console.

All folder indicators must be exactly where they are in the command: A slash after /opt/ would copy folders redundantly without overwriting, but would overwrite files. Without . the folder server-config would be created in the target path.

Switching to rsync

Unfortunately the cp command also copies a few files that I don’t want copied: repository-specific folders such as .gitea, or the folders for triggers and handlers. I only need these under server-config, not directly in opt. To fix this, I use the rsync command instead. There I can use a -u option so new files owerwrite older ones only and add --exclude to exclude files and folders that should not be copied. This looks like so:

rsync -r -u -v --exclude '.*' --exclude 'README.md' --exclude '<otherFolders>' /opt/server-config/. /opt 2>&1

But suddenly gitea no longer starts. Error message:

docker@server: [...] failed to load config file "app.ini": open: permission denied

After a long time of pondering and restarting the Docker client several times, I see that rsync has written the permissions of the source file to the target file, which was not the case with cp before: -rw-------. Now I change this by running chmod +r app.ini. Everything starts up again as usual! 🎉

Restart all applications

Since I run everything on my server in Docker, two simple commands are sufficient:

docker stop $(docker ps -a -q) 2>&1
# [...roll out config changes...]
docker restart $(docker ps -a -q) 2>&1

The finished automation

My script is now finished and simply calls the action script.

#! /bin/bash
#/opt/automation-hooks-handler/server-config-handler.sh
#[...]
 if [[ "$ATIME" != "$LTIME" ]]
 then 
 echo "--- CONFIG UPDATE TRIGGER detected ---"
 ./server-config-action.sh 2>&1
 LTIME=$ATIME
 fi
 sleep 10
done

The Script server-config-action then executes the above described actions:

#! /bin/bash
#/opt/automation-hooks-handler/server-config-action.sh

echo "--- RUN backup ---"
docker exec borgmatic sh -c "cd && borgmatic --stats -v 1 --files 2>&1"
echo "--- STOP all containers ---"
docker stop $(docker ps -a -q) 2>&1
echo "--- DEPLOY config ---"
cp -r -v /opt/server-config/. /opt 2>&1
echo "--- RESTART all containers ---"
docker restart $(docker ps -a -q) 2>&1

In order for the whole thing to work reliably, the three handler scripts must run in the background:

server-config-handler.sh
backup-pre-handler.sh
backup-post-handler.sh

They react to the respective triggers by act_runner from Gitea or by borgmatic. I will now expand the crontab accordingly and then I am done with the task for now. The console output of the entire process looks like this:

--- RUN backup ---
/etc/borgmatic.d/config.yml: Running 4 commands for pre-backup hook
Triggering container shutdown for backup.
--- RUN backup-pre-action.sh ---
Shutting down containers for backup:
gitea... 
fail2ban...
complete.
Assuming container shutdown complete. Creating the backup now.
local: Creating archive
<borg archive stats>
/etc/borgmatic.d/config.yml: Running 4 commands for post-backup hook
Triggering container restart after backup.
--- RUN backup-post-action.sh ---
Restarting containers after backup:
fail2ban...
gitea... 
watchtower...
complete.
Assuming container restart complete. Exiting.
local: Pruning archives
local: Compacting segments
compaction freed about 1.82 MB repository space.
local: Running consistency checks
summary:
/etc/borgmatic.d/config.yml: Successfully ran configuration file
--- STOP all containers ---
<container ids>
--- DEPLOY config ---
sending incremental file list
<files that are copied>
sent 206,558 bytes received 3,463 bytes 420,042.00 bytes/sec
total size is 193,167 speedup is 0.92
--- RESTART all containers ---
<container ids>

Great! Now all I need is for this log to be delivered to me if something goes wrong.

Server configuration with Git

Schallbert — Mon, 15 Jul 2024

What is it about?

I want version control for the configuration of server applications
Discussion of technical solutions for implementation
Tutorial: Exclude Docker/Gitea/other “secrets” from version control
Tutorial: Create a deployment pipeline
introduce automation trigger for subsequent rollout of files on the server
Next article: Integrate automation on the server and install upgrades

Background

I have Watchtower running on my server to automatically keep the installed distributions up to date and freshly patched. Recently I had a case where my Gitea Action Runner would no longer start automatically and even when started manually it gave an error message.

Docker had apparently updated itself without my supervision and was now throwing a volume error when starting up the runner container, which I had never seen before. The error message was clear and could easily be fixed with small changes in a configuration file for Gitea. Nevertheless, I now had the feeling that version control for my configuration would be useful for my future self in order to be able to better understand changes, updates and the reasons behind.

Preliminary considerations

It sounds like a circular reference to me: I record the configuration files for my server in Gitea, which itself runs on my server. This could become interesting with auto-deployment. But more on that later.

Two options spontaneously come to mind for getting version control with automatic synchronization:

1. Hardlink

This solution would store the configuration files scattered across many folders on my server in a folder declared as a repository using a hardlink. Why use a hardlink? Because every service on my server runs encapsulated in itself and has its own configuration files and environment variables stored together with the service. Without hardlinks, I would have to version the entire service folder and make my .gitignore correspondingly complex.

With Git, I would version the files mapped via hardlinks and make their contents available on Gitea in this way.

schallbert@server:/server-config-files –> hardlinks –> repository –> Gitea

So I would get a downstream “version monitoring” with a backup copy in the sense that the files are now lying around multiple times.

2. Repo and auto-deploy to the server

In this scenario, I keep the configuration files locally on my laptop and can version them with Git as usual. On Gitea, I would map the repository, and at the end of the chain, my runner would have to auto-deploy on the server every time the configuration changes and then restart the affected containers.

schallbert@laptop:/server-config-files –> repository –> Gitea –> act-runner –> server:/<service1...ServiceN>/config-files

I would get the administration “main” on my laptop, and the server would follow the mapping on Gitea.

Problems

In both cases, I don’t have the option of testing changed configurations in advance. Everything I do goes straight to “Prod” and would be live. In the worst case, I can easily mess up my setup.

This doesn’t change the status quo ante, where I updated the files directly on the server. So a new problem only in the sense that I didn’t have a Prod operation when I set it up in the first place and so there was no risk of failure.

Solution two seems to be more complex to implement, because I need a deployment pipeline that rolls out the files in the repository on my server. Especially since the runner is in a Docker container, while the configuration files are located directly in the server’s file system.

Direct access to the server file system is technically possible, but according to my limited understanding it would mean a large attack surface for all my services, whose configuration files could now be directly manipulated via the repository in Gitea.

Selecting my solution

Solution two at least places the immediate live problem on my laptop, so that I don’t have to mess around with the production system in the first step. I think it would be easier to set up an integration environment here with which I can check my configuration changes in advance. Since I have all of my services running in Docker, this could perhaps be solved quite easily.

Let’s get to work

Create a configuration repo

Okay, then the first step is to get the configuration files from the server. To do this, I use the file transfer command scp: scp server:/path/to/source path/to/target about a dozen times until I have caught all the files.

I set up the folder structure in this repo exactly as the files are on the server. I hope that this will make my life a little easier later.

Secrets in `docker-compose.yml`

But what do I do with “secrets” in the configuration files? Private keys, registration tokens, hashes? I would rather not have them lying around more or less openly in the repository. When using docker-compose it’s quite simple: I can store secret values in hidden files for environment variables and exclude them from Git tracking. In the simplest case, such files are simply called .env and contain a list of environment variables in the style of

BORG_PASSPHRASE="<redacted>"

In the corresponding docker-compose.yml I pull the variable from the .env file as follows:

- BORG_PASSPHRASE=${BORG_PASSPHRASE}

I make these changes locally on my laptop. I use .gitignore to specify using .* so hidden files and thus .env should not be included in the repository. But how do I know whether the containers are still booting correctly?

Secrets in Gitea’s `app.ini`

With Gitea, I’ve had a much harder time storing secrets in files. The app.ini is also written dynamically by Gitea, so the file looks a little different every time the service is restarted. After a long search, I found this issue, in which a solution for storing secrets separately was sought and found.

The only thing that seems sensible to me is storing the values INTERNAL_TOKEN and SECRET_KEY separately.
The other two properties LFS_JWT_SECRET and JWT_SECRET are automatically generated anyway and regularly overwritten.

In my configuration I only use INTERNAL_TOKEN. So I will copy it in plain text and without quotes into a hidden file (.INTERNAL_TOKEN) and make it available to the container via a Docker volume:

# Gitea's docker-compose.yml
# [...]
 volumes:
 - ./.INTERNAL_TOKEN:/run/secrets/INTERNAL_TOKEN:ro
 # [...]

In the app.ini it is now important to use the path specified in the compose file:

# Gitea's app.ini
# [...]
 server:
 INTERNAL_TOKEN = /run/secrets/INTERNAL_TOKEN
 # [...]

Not INTERNAL_TOKEN_URI=/run/secrets/INTERNAL_TOKEN as stated in the issue linked above, because this creates an error in V1.22.1 I am currently using: Unsupported URI-Scheme.

A very rough test

To check whether the changed configuration files still work, I install docker and docker-compose on my laptop. Then I try to start the containers.

# docker console log
Error: Network 'caddy-proxy' declared as external, but could not be found.

Oh right, Docker is not yet configured here. So create the network: sudo docker network create caddy-proxy and try again. The download of Gitea and its dependencies begins and the container starts - although not as I had imagined: The folder permissions within Docker are incorrect, meaning that neither Gitea nor Act-runner can access all the required files.

Nevertheless, I find the first error:

# gitea container log
WARNING: The GITEA_RUNNER_REGISTRATION_TOKEN variable is not set. Defaulting to a blank string.

I had forgotten to put the token string in quotation marks.

So for a fully functional integration environment, I have to solve at least two more problems: Folder permissions for the Main on my laptop must be set up in such a way that the Docker user also has write permissions. A simple solution for now is to append a :Z to the volumes in question and mark them as private unshared. Now the volume definition in docker-compose.yml looks like this:

# gitea/docker-compose.yml
# [...]
volumes:
 - ./gitea:/data:Z
 # [...]

I need a second file for environment variables to redirect my services to localhost. At least the service starts this way and I can see the log output. I can already spot most of the configuration errors.

I’m not sure, but I might have additional problems with the caddyserver such as certificate management, proxy settings and so on.

Create a deploy pipeline

Now it would be great if the files uploaded to the Gitea repo (and previously tested locally for functionality) would automatically find their way to my server. For this I could create another Docker volume where act-runner would then put the data stored via on:push trigger. They would then be available on my server.

Setup

If we remember my last attempts to provide artifacts on the server using act_runner, we can use a large part of that for this task as well:

# deploy-to-server.yml
# Workflow for saving the server's config repo to the local disk system

name: Upload-server-config
run-name: ${{ gitea.actor }} uploads server configuration files
on:
 push:
 branches:
 - main

jobs:
 # Deploy job
 build:
 runs-on: ubuntu-latest # this is the "label" the runner will use and map to docker target OS
 container: 
 volumes: 
 # left: where the output will end up on disk, right: volume name inside container
 - /opt/server-config:/workspace/schallbert/server-config/tmp

 steps:
 - name: --- CHECKOUT ---
 uses: actions/checkout@v3
 with: 
 path: ./tmp
 - name: --- RUN FILE CHANGE TRIGGER ---
 run: |
 cd ./tmp/automation-hooks-trigger
 touch server-config-update.txt

Strange volume errors

But the road to this point was rocky. For a long time I had only specified /server-config under volumes: on the container side and not the working directory of the runner. Then the action runs through and all commands in the #TEST section also work. But when I look on my server, the server-config folder created by Docker remains empty.

I use the following debug code under RUN FILE CHANGE TRIGGER to help me find the path errors:

# deploy-to-server.yml
# [...]
run: |
echo "hello world"
touch updated.txt
pwd
ls -al

I use echo to check whether my code is even being executed in the runner. The touch command puts the current timestamp in the updated.txt file so that I can later use this as a “hook” for further automation. pwd shows me the active path within the runner so that I can correctly map the Docker volume to the server hard drive. ls -al shows me whether the configuration files compiled in the CHECKOUT step were written correctly.

This tells me that the volume path on the “right side” was wrong. I redirect it to the active directory of the runner:

# deploy-to-server.yml
# [...]
# left: where the output will end up on disk, right: volume name inside container
- /opt/server-config:/workspace/schallbert/server-config/
# [...]

Then I got the following to read:

# gitea / act-runner console log
failed to create container: 'Error response from daemon: Duplicate mount point: /workspace/schallbert/server-config'

It seems that the runner automatically creates the “right side” of the mount point itself and therefore cannot be reassigned. Only by adding another path part, in my case /tmp - see above - I fix the error and the long-awaited folder server-config finally appears on my server 😌 with the following content:

schallbert@schallbert-ubuntu-:/opt/server-config# ls -al
total 52
drwxr-xr-x 9 root root 4096 Jul 12 15:38 .
drwxr-xr-x 9 root root 4096 Jul 11 19:56 ..
-rwxr-xr-x 1 root root 395 Jul 11 20:05 boot-after-backup.sh
drwxr-xr-x 3 root root 4096 Jul 11 20:05 borgmatic
drwxr-xr-x 2 root root 4096 Jul 11 20:05 caddy2
drwxr-xr-x 3 root root 4096 Jul 11 20:05 fail2ban
drwxr-xr-x 8 root root 4096 Jul 12 15:38 .git
drwxr-xr-x 3 root root 4096 Jul 11 20:05 .gitea
drwxr-xr-x 4 root root 4096 Jul 11 20:05 gitea
-rw-r--r-- 1 root root 312 Jul 11 20:05 .gitignore
-rw-r--r-- 1 root root 557 Jul 11 20:05 README.md
-rwxr-xr-x 1 root root 371 Jul 11 20:05 shutdown-for-backup.sh
-rw-r--r-- 1 root root 0 Jul 12 15:38 updated.txt
drwxr-xr-x 2 root root 4096 Jul 11 20:05 watchtower

Distributing the configuration on the server

Okay, that’s the first step. I now have a properly configured Git repository that shows my server configuration and can be maintained and at least rudimentarily tested from my laptop. I can also use an automatic Action to store configuration updates on the server and write an update file with a timestamp.

Now the update has to be received on the server, distributed and the affected programs and services have to be restarted. But we’ll look at this in the article Roll out server configuration.

Milling steel

Schallbert — Mon, 10 Jun 2024

For our retro gaming booth at the Maker Faire Ruhr 2024 I milled cutouts in PC cases. I would now like to tell the story behind and share suitable parameters for machining steel on “soft” hobby machines.

But be careful!

First of all, I would like to mention that I made some safety-related stupid mistakes that should definitely be avoided in the future:

Never handle rotating machine parts with gloves on!¹
Coolants and lubricants must be suitable for the intended purpose; risk of smoke and fire!
Protective and barrier devices must be firmly mounted and must not be bypassed!

So if you are dedicated to occupational and operational safety, you should not watch the video provided below.

The workpiece

I have to work on steel sheets from PC cases (MIDI towers) with a thickness of 0.7mm-0.9mm. The material is galvanized, cold-rolled steel (SECC), i.e. a cheap and relatively easy-to-work type of steel with a low carbon content and a small admixture of other alloying elements.

This type of steel is used for sheet metal in automobile construction and many household appliances, especially in “white goods”, and is also often used in vending machine and switchboard construction. This milling work can therefore be transferred to various other areas of application.

The dimensions of the housing sides are between 35x45xm and 52x50cm, so they all fit on my machine. With one exception, the side panels lie flat with their visible side and without cutouts, so my vacuum table can easily be used to hold them down.

The side wall of a housing has gill-like perforations for ventilation and a recess for a handle. For the grille I use cling film to maintain the vacuum pressure, while the loss of vacuum is accepted when milling the handle recess.

CAD/CAM

The cutouts to be made are all geometrically simple and take into account r=1.5*milling radius for curves. The window for the “sea PC” is designed in two parts and is intended to represent a storm scene at sea.

The window is 0.25mm smaller all around than the recess, which results in an almost play-free fit and looks very professional.

In the CAM I set up counter-rotating milling, which is advisable for less rigid machines like mine when machining hard and tough materials. The machine runs a little smoother and quieter this way - but it still shakes and vibrates considerably.

For the depth of cut, I choose a value that is four hundredths of a millimeter less than the material thickness. This way, I practically only leave a wafer-thin film of paint and corrosion protection, but I don’t wear out the adhesion promoter mat.

Milling parameters

Compared to machining wood, the cutting speeds for steel are significantly lower. While I can work with 450m/min in birch multiplex, various cutting data calculators like this one recommend values between 80m/min and 200m/min. This results in a parameter set like S4250u/min, F255mm/min for my two-cutter made of micro-grain carbide, for example.

This cutter has cutting edges bevelled at a 45° angle of 0.2mm in length and a helix of 30°. To increase the service life, it is coated with a TiAlN alloy approximately 3µm thick using the vacuum process PVD.

After consulting the cutter manufacturer, I preselected parameters on my machine, which I then adjusted in test runs and was thus able to achieve a higher depth of cut at a lower chip temperature (no more blue chips). Here are the values that I ultimately used for machining:

Milling cutter	Cutter geometry	Speed	Feedrate XY	Feedrate Z	Depth per pass
6mm, Solid Carbide	2Flutes 30°, l=12, 45°0.2Fase	6000u/min	550mm/min	250mm/min	0.8mm

When machining steel, pay attention to beveled cutting edges for the milling cutters
Increase the feed rate, reduce the depth of cut (especially applies to “soft” machines)
Dry milling is less problematic than I thought

This Youtube video by Stefan Gotteswinter explains very clearly the advantage of lower chip thicknesses when using beveled cutting edges. This works best when the depth of cut is not greater than the length of the bevel at the end of the cutting edge. I am also advised on cnczone to choose a “high-speed milling” approach.

For my two-cutter, I adjust the cutting data so that I drastically reduce the depth of cut, but increase the feed and speed:

Milling cutter	Cutter geometry	Speed	Feed XY	Feed Z	Depth per pass
6mm two-cutter	2Flutes 30°, l=12, 45°0.2chamfer	10600rpm	970mm/min	450mm/min	0.2mm

I am really looking forward to testing these values in practice.

Preparing the milling process

As this is my first time working with steel, I had to run a few errands and try things out beforehand.

Cooling lubricant

Cooling is essential here to protect the milling cutter, says the milling cutter manufacturer. So I got myself a syringe with which I can apply lubricant precisely. But: which product should I choose?

For just three parts, I don’t buy a minimum quantity lubrication or cooling lubricants from industrial supplies. So I try out products available at home:

Spray lubricant WD40: Not suitable for me. Evaporates quickly, smells.
Sewing machine oil / fine oil: Not suitable. Very liquid (flows into vacuum holes), smokes and stinks.
Rapeseed oil for frying: Wonderful! Lubricates well, relatively viscous, low odor, hardly any smoke development.

Chip protection

The metal chips created during processing are short, hot and sharp-edged. They tend to get caught in clothing and fly so far that even at a distance of more than a meter from the machine, plenty of chips still arrive.

So I sawed off a segment from an acrylic glass tube with a diameter of 20cm, slotted it and hung it on the Z-axis as a chip protection.

My mistake here: I relied on the Acrylic’s clamping force alone and did not additionally attach the chip protection with screws. Fortunately, it only jammed when the last part was being made.

Milling

You’ll notice immediately that the machine does not have the necessary rigidity to process steel. As soon as the milling cutter plunges into the material, the machine begins to rattle and shake. Nevertheless, it bravely fights its way through the material, while I am always ready with a drop of oil at the milling path.

Milling Steel!↗

The spindle has more than enough reserves available even at the low speed. However, when positioning the workpiece, I make sure to work as centrally as possible on the ball screw of the Y-axis. This is where the machine experiences the least torsional forces. The chips look good to my eyes.

The noise changes when milling: I have more vibrations in the machine when I mill in the Y direction than when only the X-axis moves. I will look into this phenomenon in more detail later.

The result

After machining on the CNC, I can simply cut out the windows with a cutter knife.

Your browser does not support the video tag.

Video: Cutting the cutout with a cutter knife

I quickly remove the (minimal) burrs with a file.

Then I make windows out of transparent acrylic glass, thickness 3mm, which I provide with a 10mm wide fold at depth of 1mm all around. This takes into account the thickness of the double-sided adhesive tape for fitting and the pane sits flush with the PC case.

The picture above shows a Celsius 1000 Workstation case from Siemens, which now looks really classy with a simple window.

Gloves can be torn away if they accidentally come into contact with the milling cutter. This can result in serious hand injuries. ↩︎

CNC Tuning #3

Schallbert — Sat, 04 May 2024

This is the final part of my series on optimizing manufacturing time on CNC milling machines. It focuses on the path interpreter of the CNC control software.

The control software path interpreter is responsible for reading the instructions from G-Code and translating them into machine movements. For example, if G01 X100 Y200 Z-2.0 F2000 S18000 is to be executed, the interpreter must take into account the kinematic parameters of the machine: which ramp is used to accelerate to the target feed, whether the desired feed is even permissible, and from which point braking must be carried out again in order to reach the target point exactly. The data obtained in this way is finally translated into a step sequence for the three stepper motors, loaded into a buffer memory and finally output to the motor amplifier stages in real time.

Step 3: Optimizing the path interpreter

Many control software providers have programmed integrated milling path filters, which can be configured using the G64 command or via the menu. I’ll link a few examples from different manufacturers here, where you can read about specific details:

The trick now is to find a configuration that suits both milling job and machine kinematics. It’s always about a compromise between production speed and path tolerance. If you want to run at high feed rate, sharp edges are actually rounded, connected line segments are approached within a tolerance band or paths are simplified so that path points outside the acceleration that can be achieved by the machine are not just a pass-by.

Basics: What is Look Ahead Feed?

This means that the path interpreter of the control software is already looking at the next operation(s) while executing the current milling operation - the algorithm checks how it must brake and accelerate again ahead of time and how to best interconnect, interleave or filter path segments. I explain the physics behind it in the CNC setup article.

This can save a lot of time, the milling process becomes more fluid and the machine experiences less stress. In addition to avoiding acceleration and braking processes, parameter G64 can be used to set whether and how milling path filters are allowed to change the planned trajectory in order to further optimize for higher path speeds.

Ideal for engraving operations

Such options can be used excellently in engraving operations: higher tolerances are usually OK here, and slightly rounded edges are often acceptable. With 3D engravings, milling through the many paths takes a long time, and a lot of time can be saved here.

Since I use EdingCNC, I can only give tips for parameters of this control software. Now let’s go into detail about the settings!

Milling path tolerance `G64 P*`

This parameter specifies the general tolerance of path accuracy to the control software.

I have set my machine so that it allows deviations from the specified milling path of up to LAF blending tolerance = 0.3mm in engraving mode. Alternatively, the configuration can be done using command G64 P0.3. In the picture I have sketched how such a setting can work in practice: The milling path (white) consists of G01 line segments. The tolerance is marked with the help of circles. The path interpreter of the control software can now independently define circle segments so that the highest possible path speed is achieved within the tolerance.

The path determined in this way (turquoise) resembles a spline curve. It should be noted that all lines can be affected by this setting - even those that have strict tolerance requirements. If this is not desired, a smaller value should be used here.

Path rounding `G64 R*`

If two line segments meet at an angle of up to LAF full speed blending angle threshold = 12°, the machine will try to follow the angle exactly without braking. For larger angles, travel is carried out at reduced speed with a tolerance defined by G64 P*. This configuration is programmed with G64 R12.

The values here must be set with great caution because they lead to acceleration peaks on the machine. Large values can cause step losses and increased wear on the CNC milling machine. To achieve this, the path speed and accuracy are maintained optimally.

Path rounding at reduced feed `G64 S* D*`

This configuration gives the control software the ability to round edges up to the set tolerance value: LAF reduced speed path smoothing delta = 0.001mm. In combination with the rounding, the angle up to which this rounding should be carried out must be set: LAF reduced speed path smoothing angle threshold = 0°.

This function is best programmed in combination with path rounding at full feed like so: G64 R* S* D*. The advantage here is that line segments are followed very precisely as shown in the picture, and at the same time the edges are rounded off in a finely adjustable way. Up to the angle defined via R, the path is followed exactly, even at full feed.

If very large angles are set at S, there will be greater rounding corresponding to D. However, if the angle specified in the milling program exceeds the configured value, the algorithm falls back to the values stored in P or to the exact path following mode.

Path simplification `G64 Q*`

If the G-code file contains numerous, very short line segments (as is often the case when creating paths from image files), this function can help by simplifying the milling paths. The number behind the parameter indicates the tolerance up to which line segments are merged into a line.

This setting relieves the load on the machine’s buffer memory and at the same time increases the average path feed rate. I think this setting is particularly interesting for engraving operations when working with “noisy” data and there was no option in CAD to reduce the number of paths.

Acceleration filter `G64 F*`

Using these parameters, the acceleration peaks that may be generated by Look Ahead Feed can be mitigated. The minimum value to be set is determined by the quotient of the selected feed for the operation and the maximum speed of the machine.

Example: Fmin = Foperate / Fg00, with Fg00 = 12000mm/min and Foperate = 5000mmm/min, Fmin = 0.42

So at a feed rate of 5000mm/min the maximum possible acceleration of the machine is called up and the curve radii are set so that they are ran at a constant path speed. All values greater than or equal to Fmin = 1 are inherently safe, regardless of the selected feed rate.

This parameter was introduced for better processing of elastic materials such as rubber or cork, where a constant feed speed is critical and path stability is of secondary importance. The parameter is used in combination with G64 R*. The following programming for the example above would call up the full capacity of the machine: G64 R120 F0.42, whereby F5000 is used, F12000 is the maximum feed of the machine and acute angles up to 120° are taken without braking at all.

If I apply these values to my machine, the image shown opposite is created. A rectangle with an edge length of 25 x 100mm should be made here. The target trajectory is again shown in white. The turquoise curve shows the curve with a constant path speed that was driven by programming the acceleration filter. The lower the feed in relation to the maximum feed and the higher the acceleration of the machine is set, the smaller the necessary radius becomes.

However, in my experience, this parameter is not used for engraving and milling solid materials.

My configuration for the Hobbyline 4530

I played around with the various LAF settings at the cost of a lot of blanks to learn and improve parameter settings. I achieved the shortest production times by previously optimizing the kinematic parameters with the following configuration:

G64 Q0.3 R12 D0.3 S120

With that I tell the machine the following:

“Use the look-ahead feed algorithm and try to merge short line segments less than 0.3mm in length. Try not to reduce feed rate when two line segments are at an angle of less than 12°. At larger angles up to 120°, rounding can be done with up to r=0.3mm tolerance. For even larger angles, the specified path must be followed. If necessary, brake to a standstill.

Conclusion

Since the parameter optimization for engraving operation took me quite far away from the parameter set for normal milling operation, I simply installed my control program a second time.

There I was then able to specify the speed, axis acceleration and path parameterization separately. Now I have the best of both worlds on one machine:

A very precise, smooth milling operation with the following parameters:

Fmax XY 12000mm/min
Fmax Z 4800mm/min
Amax X 1000mm/s²
Amax Y 1000mm/s²
Amax Z 1000mm/s²
G64 P0.1 R6

And via the other link an extremely fast engraving operation:

Fmax XY 12000mm/min
Fmax Z 4800mm/min
Amax X 4000mm/s²
Amax Y 3200mm/s²
Amax Z 1600mm/s²
G64 Q0.3 R12 D0.3 S120

And where is the proof?

The embedded video shows the same engraving project with four different levels of optimization:

top left: stock configuration of the machine, standard milling paths from CAD/CAM
top right: Speed and acceleration increased to “milling mode”
bottom left: Look Ahead feed switched on G64 P0.3, milling paths optimized in CAM
bottom right: Optimized for “engraving mode” as described above, CAM flyover reduced to 2mm and empty runs minimized

Tuned Engraving: Compare↗

CNC Tuning #2

Schallbert — Sat, 27 Apr 2024

This is the second part of my series on optimising production time on CNC milling machines. It focuses on the kinematic machine parameters for engraving operations.

Step 2: Tuning the kinematic parameters of the portal milling machine

I have explained the theoretical basics in detail in the article on CNC control configuration. After many tests and consultation with the manufacturer, I recorded my problems with it and the associated solutions in Conclusion on one year of CNC operation. The following sections apply the above findings to a demonstration machine that I was able to borrow.

Travelling speed in rapid traverse

For the demo machine, I was able to increase the rapid traverse speed from the suggested Fxy = 4m/min and Fz = 2.4m/min to Fxy = 12m/min and Fz = 4.8m/min. If I set all three axes in motion simultaneously, I get a combined step frequency of 96kHz due to the pitch of the ball screws and the microstep configuration (8 per full step) of the motors, which is below the maximum step frequency of 125kHz of the controller.

Acceleration in rapid traverse

The manufacturer’s specifications recommend aXY = 300mm/s² and aZ = 200mm/s².

I increased the acceleration in several steps and ended up with aXY = 1000mm/s² and aZ = 400mm/s² in a first round. In my experience, these values are easily possible for this small and light machine with a travel of only 30x45x14cm for normal operation. They greatly increase the production speed, because every change of direction, including those in the material grip at G01, G02, G03, are now carried out faster by a factor of 3 / 2.5.

I have often operated the machine this way and I have also driven stably and safely with these values at the maker faire. The following video shows the milling machine engraving the letter “e” with values specified above.

TunedEngraving: Teaser↗

After that, however, I wanted to know how far I could take it. True to the motto described above

“If you don’t cross the line, you don’t know where it is”

I first tested what massive step losses sound like on this machine. So I blocked one axis with a wooden beam and carefully ran into it:

Your browser does not support the video tag.

Massive step loss soundcheck

The clearly audible chattering occurs when the motor starts up, but several full steps are lost at target speed. If the rotor is already “lost” when the motor starts up, the motor only chirps without setting the axis in motion at all. If the motor only loses a few steps, but is at target speed, there is a rough and scratchy sound.

I chose the X-axis for the experiment because it is the easiest to lock. I also blocked it in the X+ direction. Reason: this machine does not have a professional fixed bearing to absorb the compressive and tensile forces of the ball screw. Instead, compressive forces in the direction of the stepper motor are safely absorbed by a flange mounted on the spindle, which presses against ball bearings located in the portal cheek.

Tensile forces, on the other hand, are mainly absorbed by the motor bearings - I didn’t want to take any risks here. So now the noise caused by sudden and jerky step loss is known - overclocking can begin!

My approach:

double the acceleration of one axis of the machine
jog around the axis in question, listening for strange noises and observing the axis movement
repeat the previous steps until massive step losses occur
reduce acceleration to the average value between “works” and “does not work”
repeat the previous step until the machine runs smoothly
perform all steps for the other axes as well

This is how I ended up with aXY = 4000mm/s², aZ = 1600mm/s², which is unbelievable for such a hobby machine. This is now an order of magnitude higher than the recommended values.

The small milling machine also seems to run stably here and the engravings produced look good. However, I could not imagine that I was still operating the stepper motors, driver stages and power supply unit within their specifications. That’s why I took measurements.

Electrical measurements

The power supply unit is specified with 3.7A at 36V. I must therefore not exceed this value permanently, neither in rapid traverse G00 nor in material engagement. As explained above, the rapid traverse in engraving mode is decisive due to the lower milling forces, so I have to pay particular attention to the simultaneous movement of several axes.

The motors can each handle a continuous current of 3.0A, so they don’t limit me here. In the electrical tests, I only check whether the energy consumption of the system remains within limits.

Test setup

I connect the multimeter known from this article directly to the output of the power supply unit and record voltage and current. For the experiment, I simulate the production of the engraving linked in the video above and in two runs first carry out a temporally coarse-resolution power factor analysis (2Hz), followed by a fine-resolution current determination with 20kHz.

Results

The figure above shows voltage, current, effective and reactive power. It can be seen that the reactive power (green) always shows short peaks when several axles accelerate into rapid traverse at the same time. The power consumption of an axis in G01/G02/G03, blue is on average around 45W, similar to that of constant speed in rapid traverse.

The power supply unit is able to keep the voltage orange fairly constant and even under full load only drops by just under 0.2V. Only the current value looked a little too smooth to me. At this point I suspected that the time resolution was too low and decided to repeat the experiment focussing on the current.

And indeed. The higher temporal resolution shows that the current consumption during acceleration of several axes shows peaks that are higher than the values already known.

Movements on the Y-axis, which must set the entire portal in motion, are creating current spikes, especially when the other axes are still moving or have just stopped. This is related to the stepper motor drivers. They continue to brake the motors at full power for about 0.5s after stopping. The resulting overlay makes the tips look even more dramatic.

Peak load benchmark

Well then let’s take it to the extreme with an acceleration benchmark. Will the power supply be able to maintain the output voltage even under full load while accelerating all motors at the same time?

To test this, I wrote G-code that makes all axes simultaneously accelerate to maximum speed and then decelerate to zero again. Then it will return to the starting point under the same conditions and repeat the whole thing.

In this way I can find out whether the power supply has enough reserves to handle load peaks and at the same time see whether the motors are already losing steps to a large extent. On average, the applied power is 95W and the supply voltage drops by 0.28V to 35.72V. The power supply must deliver a peak of 5.14A, which is almost 40% above the rated current.

Your browser does not support the video tag.

Benchmark

If you listen closely, the machine sounds a bit rough right at the start of the benchmark, but then it settles down. Lost steps can sound like this too. It is therefore very important to monitor the machine with all your senses.

Mechanical testing

Now for the final test: During the rapid traverse benchmark, I put additional load on the axes by pulling on the respective axis with my arm while the program is running. The Z and X axes continue to run stably. However, with the portal axis Y I notice that the machine now sounds different. Somehow rough, as if you were lightly dragging a rasp over a wooden board.

If I apply a large load, the stepper motor on this axis no longer even starts, but just chirps. It can no longer follow the pulses of the power amplifier. So I now reduce the acceleration step by step until the motor can accelerate the portal axis repeatably and without audible step losses, even under medium load. Here I end up with a still fast [aX = 4000mm/s² aY = 3200mm/s² aZ = 1600mm/s²].

In a final step, after repeating the benchmark five times, I carry out another reference run and look at the control software logs. All axes are now unobtrusive in terms of step losses.

Appeals for CNC tuning

Please be careful when tuning your machine. Get it to know properly before you start. Listen carefully and optimize axis by axis one after the other. Take time for tests, verifications and experiments.

Even if you have a portal milling machine that is very similar to mine, other dimensions make a huge difference in terms of stiffness and therefore possible travel speed and acceleration. Even the state of maintenance, smoother shafts, different guides, etc. can mean that you end up running different values than me.

No professional equipment is required to explore the limits of the machine. Not even a dial gauge, since step losses can also be determined with sufficient precision using the reference switches.

All I found from the electrical measurements above is that the manufacturer did a good job designing the machine and no component is undersized. On the contrary, this machine is ideal for overclocking.

And now on to the last Part 3: Optimize the path interpreter.

CNC Tuning #1

Schallbert — Sat, 20 Apr 2024

This is the first part of my series on optimising production time on CNC milling machines. It focuses on the use of CAD and CAM tools to generate more efficient G-code.

There are various measures to reduce production time with a CNC machine. Firstly, however, it should be analysed why this is desired and what possible disadvantages result from it.

Why at all?

From a commercial point of view, shorter production times ensure a higher throughput of parts and thus enable a higher turnover in a given period of time. In addition, the investment in the machine is amortised more quickly.

As a hobbyist, shorter production times are particularly interesting if the aim is to produce very complex geometries or when performing real 3D-milling. Here, machine times can easily amount to several hours per part and the potential savings are therefore particularly large.

For me, as in teaching, the scientific idea takes centre stage: I want to find out where the limit of the machine lies. I want to know why it is there. I will try to operate it stably within its limits. Still, wear and tear must be kept under control.

However, in order to know this limit precisely, it must first be crossed. How else would I know where it is?

What are the possible disadvantages?

Certain components of an “overclocked” machine could wear out faster than usual. These include power supply units and motor output stages, for example, which have to cope with higher transient currents. Bearings and guides are subjected to greater stress due to higher accelerations. Rotating spindles and machine frames have to absorb the resulting stronger and more rapidly changing forces and vibrations and may show signs of fatigue or loosening connections earlier.

In extreme cases, the machine can even be damaged when the limits are reached: There is a high probability that step losses¹ will occur beyond the limit. This is not a problem in test operation. However, the assumed position then no longer matches the actual position and there is a risk of a “crash” - if care is not taken. Here is a pretty good Youtube video by gammaflow on the subject of avoiding step losses.

If the milling paths are changed in CAD or CAM in favour of shorter production times, this can also have negative effects on cut, edge and surface quality.

A compromise must also be made when optimising the path interpreter of the control software: Higher production speed is bought with lower accuracy and/or higher machine wear.

Which measures are not discussed?

I will not consider here the optimisation of the feed rate F and speed S of the selected cutter during machining. These parameters depend on the material to be machined, the endmill geometry, the capabilities of the machine and the courage of the operator, so I cannot suggest any generally applicable tunings.

I must also exclude another major influencing factor here: The machining strategy of the CAM program in relation to rapid traverse. My CAM program does not allow me to intervene very deeply, so I always have to work manually in the G-code to optimise the last few percent of production time, which is correspondingly error-prone.

Tuning in three steps

In my optimisation, I proceeded in three steps, which can be implemented independently of each other. Each step brought me clearly noticeable improvements. This article focuses on engraving operation for several reasons:

Engraving usually allows for greater tolerances than other modes of operation. This allows greater rounding, which leads to lower accelerations, enabling higher feed rates on average
The milling forces to be expected are lower: No cutting all the way through, hardly any energy required for chip displacement, smaller chip volume than with end mills
This allows the machine to be operated closer to its mechanical and electrical limits
More optimisation potential due to larger G-code files on average

Step 1: Better G-code

This section follows the credo Garbage in, garbage out. In other words: If my G-code is already bad, the machine can never achieve optimum production times. So what characterises “bad” G-code in terms of production time?

Excessively complex and long, possibly file sizes in the megabyte range
Extremely many interpolation points and a lot of short line segments
A bunch of “empty runs” in rapid traverse on XY with the Z-axis raised
Many retractions of the Z-axis to safety height
Frequent machine waiting times (spindle run-up, coolant pump, controller buffer full)
Excessive superfinishing even with B surfaces

But how should I simplify my G-code?

Short question, long answer: It starts in CAD and may not end in CAM.

Example CAD

Let’s assume we want to mill some letters. Our CAD program has a bunch of great fonts for this. We now create a DXF file for the CAM. In the CAM, when we zoom in, we suddenly see that all the letters consist of very short line segments instead of circle segments or splines. The result: The CAM will generate G01 movements, i.e. “linear moves” or a sequence of lines.

The connected paths require the machine to change direction very often. Consequently, it must always decelerate before the connection point and then accelerate again. The machine cannot therefore travel at a constant speed, as this would require infinite acceleration at the bend point.

There are solutions to this problem, which are ideally used together:

Set the CAD programme to create splines or curves instead of line segments whenever possible
Simplify the drawing by reducing the number of support points
Increased use of radii and splines at edges, avoidance of angles

In my case, the first option was not available, so I now use a different programme for engraving fonts than for my usual drawings. As a result, I can now express lettering in line and arc segments. The milling time required for this is halved and at the same time the G-code manages with fewer lines. In addition, the cuts become rounder and adheres even more closely to the initially selected font.

Gimp and Inkscape provide good examples of simplifying paths by reducing vertices - I have already shown this for Inkscape in my article on image-to-path conversion. With just a few clicks, you can obtain drawing data at the expense of accuracy, some of which can be produced much faster.

As an end mill cannot produce a pocket with r=0 in the corners anyway, I have got into the habit of rounding off all corners and edges in CAD. Not only do parts produced in this way fit better in the hand, they are also quicker to produce - the time savings add up, especially on slow and sluggish machines.

Example CAM

In CAM, I can save a little time by increasing the use of approach points and ramps. I usually place them on the shortest line segment and thus save one braking/acceleration process. With the plunge type, the advantage of “ramp” over “helix” is that all paths of the cutter are effective paths and I can drive through without additional axis accelerations. Depending on the ramp gradient, however, the milling path is longer as overlaps occur.

The clever selection of clearing infeeds and strategies for pockets to suit the material, milling cutter and desired surface finish has a much greater effect on the production time. In many cases, trochoidal milling² produces longer files and takes more time on the machine than the classic “lawnmower”, which simply files the pocket. For the clearing infeed, I like to choose large values above ‘60%’ for materials such as wood and acrylic glass - unless the bottom of the pocket has to meet particularly high quality standards.

In my opinion, it is quickest for engravings not to use a second cutter for pocket clearing. Instead, use an engraving bit with a flattened tip - similar to those for my tests with Dibond. This takes longer to clear, but I can compensate for this by increasing the engrave depth and a fairly wide flattening of 1.5mm. I use a clearing infeed in XY of 1mm with such a milling cutter. What remains is the time advantage due to fewer tool changes.

But how can I reduce the empty runs?

At last a few crisp, short answers:

Lower retraction height in Z (I use 5mm but 2mm would probably also work for me)
Optimise the milling strategy in the CAM to “minimum paths”
For nesting: Make sure that the geometry is not produced line by line, but part by part.
If in doubt, optimise the G-code by hand. Worthwhile if the CAM program cannot work well enough and many identical parts have to be produced

And where is the problem with retractions of the Z-axis?

Retractions occur very frequently, especially in engraving operations
the loss of time adds up
When re-engaging, the material is usually not moved at rapid traverse but at the speed intended for the milling cutter with G01.

Good planning of the milling sequence in the CAM can bring an improvement here. As many sections as possible should be connected and produced in one go without retractions. In CAM, rapid traverse should be used for retraction and re-entry in ‘Z’.

How do I reduce waiting times?

Cooling lubricant switching processes, spindle speed changes or tool changes are associated with waiting times that can usually be easily reduced:

Allow minimal quantity lubrication to flow during the whole job sequence
using as few tools as possible at the same speed for a product
Use stops to simplify machine loading and avoid having to set zero points again and again
If the machine is operated with completely different parameters for engraving and milling operation, it is advisable to install the control software twice with the appropriate parameter sets in each case. This eliminates the need to always change the configuration.

Continue in Part 2: Turning up kinematic parameters.

Step losses occur when the axis of a stepper motor no longer rotates synchronously with the step sequence given by the controller, but instead travels fewer or more steps than dictated. Step loss is almost always caused by inadequate system design (e.g. insufficient or unstable supply voltage, output stages that are too weak or not adapted to the motor) or overloading of the stepper motor. Rarely, however, step losses can be symptoms of defects such as bearing seizure, winding short circuit or blown buffer capacitors. ↩︎
Trochoidal milling is not a bad practice. It has other strengths, but is rarely useful in engraving. ↩︎

Vacuum clamping

Schallbert — Tue, 02 Apr 2024

As I mentioned in the Maker Faire Ruhr 2024 article, I had major problems clamping my workpieces for the production of spirographs.

In this article, I would like to go into this in more detail, analyze the causes and finally present a few possible solutions.

What I can anticipate: All problems can be solved with knowledge of the causes and a little time, planning and material expenditure.

The problem

As usual, I had defined the path planning for the spirographs in such a way that I first had the inner areas (pockets, parts, holes) milled and then the outer areas. Cut-outs and chamfers were to be produced at the very end. I also planned to machine the parts on both sides so that the backs of the gears would also have broken edges and smooth transitions.

Vacuum loss

In my first attempt, I proceeded as I had already successfully done in the spirograph project with the material acrylic glass: The back is machined first, but not milled all the way through. After turning the workpiece, only the chamfer cutter should be used to cut out and break the edges at the same time.

This worked perfectly well for the first clamping.

Your browser does not support the video tag.

Back machining of the spirographs.

With the second one, however, so much negative pressure was lost when the surface was pierced that the material detached from the vacuum table and took on its original, slight curvature. So I had to stop the milling process and think of another strategy to hold it down.

At this point, I couldn’t really explain why so much more pressure was being lost than during production in acrylic glass.

Cling film

I tried to counteract the pressure loss with cling film. This resulted in several consequential problems: If the cutter passes through the film, it is caught by the cutter when milling closed contours and wrapped around it at lightning speed. The cutter no longer cuts, but only rubs against the material, which very quickly leads to heat marks on the wood.

But even if I attach the cling film in advance: As I cut out parts, the local negative pressure on the part is already so small when it first enters the groove already present on the underside that it can hardly be held in place.

Twisting of the workpiece

For the second attempt, I adapted the strategy: The end mill would mill through the workpiece, so I didn’t need a second set-up. I wanted to optionally realize the edge breaking on the back side later via a holder clamped on the table, into which I would be able to insert the parts.

While cutting out the parts still worked without any problems (again, milling the “inner” parts first), I ran into major problems when applying the chamfer to the front. The gears began to twist and the template with the ring gear could not be held down at all.

Too little time

I therefore had to resort to using holders specially made for the workpieces when machining the front side. Their design and manufacture took time that I had not planned for at this point.

Before the trade fair, I wasn’t able to do an in-depth analysis - a quick solution had to be found.

The causes

With a few weeks’ hindsight, I can now say that several factors led to the problems. A difficult geometry, unfavorable material properties and certain peculiarities of the vacuum table design all came together. But let’s get to the bottom of it all here.

Workpiece

The raw material is medium density fiberboard of 10mm thickness. It is covered on one side with 0.75mm thick olive wood veneer. This material is inherently problematic for processing on a vacuum table:

Due to veneering and storage, the boards are slightly curved¹, so they want to stand out from my vacuum table.

MDF is permeable to air, so the vacuum pump has to deal with larger leakage currents than usual (I only get 300mBar when clamping the panel instead of the usual 800mBar without leakage currents). The low material thickness intensifies this effect.
MDF is not only permeable to air vertically to the panel plane, but also horizontally. This greatly reduces the holding forces in the vicinity of milled grooves, as air can now also penetrate the material laterally, thus additionally increasing the leakage currents.

The video linked here Youtube: Displacement force measurements by Anton CNC clearly shows how different materials behave on a vacuum table with a porous surface. Also highly recommended on this topic is this video YouTube: Determining holding force during machining by vakuumtischDE to get an impression of the load the workpiece can be under during the milling process.

Geometry

My design for the spirographs is not particularly easy to hold down:

The template with ring gear has thin walls of only about 2cm thickness. They are thickest on the left and right sides, but have a large recess in the middle through which air can penetrate the material in all directions, thus reducing the holding forces.
The gear wheel has holes to accommodate pins. The contact pressure is reduced around the holes.
The teeth of the ring gear and gearwheel are so exposed that they can make virtually no contribution to the holding force due to cross currents of air in the material. At the same time, however, the tooth heads provide a large lever for twisting or shifting the workpiece: an unfavorable pairing.

If I take this as a basis for practice, only the vacuum holes inside the circle in the image above are capable of generating any significant clamping forces. However, these are again close to the center and therefore offer little resistance to twisting of the workpiece. In addition, the gear wheel here has several holes and therefore additional “leaks”.

Milling forces

The forces introduced into the material by the tool (the milling cutter) always act in the XY direction and, in the case of V-cut milling cutters and end mills with spiral/twist, also in the Z direction.

Forces due to feed: The faster the material is moved through, the stronger the forces in the XY direction act on the material in the direction of travel.
Forces due to cutter rotation (conventional or climb-milling): If an end mill without twist is used, forces are generated orthogonally in the plane of the traverse direction. So if I move in the positive X-direction, the milling cutter will also exert a force in the Y-direction, which is introduced by its own rotation: In conventional milling operation, the cutter wants to avoid the material, so it is deflected towards Y+ (as clearly shown in this article). In the opposite direction, the deflection direction is inverse and the cutter wants to deflect towards Y-.
Forces due to cutter geometry: If the cutter has a spiral or is angled in its geometry, additional forces act in the Z direction. With Vcut and downcut cutters, the material is pressed onto the machine bed, whereas with upcut spiral cutters, the chip is ejected as desired. However, the workpiece also experiences a force that can lift it off the machine bed in extreme cases.
If milling is carried out in the full groove (i.e. the milling cutter is engaged over its entire diameter instead of moving along the edge of the workpiece), the forces transverse to the milling direction increase even more: In the image, it moves in the opposite direction at the upper edge of the groove, which pushes the milling cutter “away from the material”, i.e. downwards in the image. At the same time, it moves in synchronization at the lower edge of the groove, which pulls it “into the material” - also downwards.

Vacuum clamping system

As described in the articles about vacuum pumps and vacuum tables, I have a dry-running rotary vane pump connected to a perforated grid vacuum table. When processing porous materials, this combination is just as problematic as grid vacuum tables with small-volume pumps:

My hole-grid vacuum table has blind holes with a vacuum channel of 0.5mm diameter in their center. This constriction relieves the pump when holes are uncovered, as it does not have to provide the full air flow that the 5mm bore allows. However, the hole can build up less negative pressure due to its limited air flow in combination with MDF.
Due to the high leakage currents, my vacuum pump is at its limit with its volume flow. Other pump types are much better suited here (see radial blower, side channel compressor).
A vacuum table generally only clamps downwards (in the Z direction). However, forces in the XY direction must also be absorbed to prevent the workpiece from shifting or rotating. These are coupled to the vacuum on the workpiece via the coefficient of static friction µ (and can be influenced by material pairing and surface properties), but if the latter is too low, the workpiece is not held reliably.

Although the vacuum channels are located in a 10mm grid, the distance relevant for the leakage flow is only 5mm. This is because the edges of the blind holes (each 5mm in diameter) are at this distance from each other, so that the air flow has to travel a shorter distance across the material than through it.

Comparison with the resistance network

The actual clamping force can be easily estimated by imagining the pump, vacuum table and workpiece as an electrical network. I have prepared a model for this and present it below.

Resistors: Vacuum table, workpiece

The suction holes of the vacuum table represent resistors for the air flow, which are connected in parallel. The workpiece is again connected in series as a resistor, which is connected with its other pole to “ground”, i.e. the ambient air pressure.

Vacuum manifold = switch, pressure gauge = voltmeter

The pressure gauge in the suction channel can be seen as a voltage meter. I take the vacuum distributor with its valves for switching the vacuum sections as an electrical switching device. If the switch is closed, it establishes the connection between pump and vacuum table. If it is open, it blocks the air flow.

Vacuum pump = voltage source

In this picture, the pump represents a voltage source, which in my case can provide a maximum of U = 8V corresponding to -800mBar. Due to the internal resistance of 0.2Ohm present in real voltage sources, it can generate a short-circuit current of I = 40A, analogous to the 40m³/h flow rate. This comparison is of course not physically correct, because I would have to work with mass flows instead of volume flows. In addition, the flow rate is not linearly related to the pressure everywhere in the characteristic diagram and the resistance values I have assumed are only reference values. In my opinion, it is nevertheless suitable for modeling the situation under steady-state conditions.

With workpiece materials such as acrylic glass or metal, the air flow is close to zero, so their “resistance” is extremely high. This causes most of the tension (negative pressure) to drop here, so that the workpiece is firmly clamped.

When using airtight materials, all is well with the world: the pressure gauge correctly indicates the pressure on the material when the workpiece is still unprocessed. Even in the case of milled holes, only the resistance of the material “in series” to the affected suction holes is set to zero. This means that the negative pressure and therefore the holding force on the part itself is hardly reduced.

If you look closely, you can see that the pressure gauge measures the pressure difference upstream of the vacuum manifold. Its value is therefore only meaningful for the clamping force if the workpiece has a very high flow resistance.

Special case of porous materials

The situation is completely different with porous materials such as MDF:

Here, the contact resistance of the material is only just an order of magnitude higher than that of the vacuum holes. I determined the values through tests on the vacuum table. They are based on pressure measurements, calculations of the workpiece surface, and the number of active suction holes covered by the material. Two effects should be particularly emphasized here, which made holding down even more difficult for me:

Pay attention to the resistances in the material in parallel to the contact resistance in the Z direction. They are caused by the fact that the air can now also move through the material “between the holes”.
On the right-hand side of the picture you can see what happens when the workpiece protrudes into an inactive vacuum area: The suction holes are connected to each other with virtually no resistance by means of vacuum channels in the table. As the workpiece allows “transverse air flow” in the XY direction, significantly less resistance has to be overcome by connecting the suction holes than for penetration in the Z direction. This means that only R29 and R22||R21 are still in series. If the workpiece covers several inactive hole fields, R22 approaches 0.

As a result, the resistances upstream of R19, namely R18||(R29+(R21||R22)), become so small that the pump practically only draws ambient air here and can hardly apply any clamping force at this point. The cross air flow also affects the pressure drop of R17, R15 via R25.

Now the small suction holes act as voltage dividers, so that a not inconsiderable part of the negative pressure “sticks” to them.

This means that the pressure displayed on the pressure gauge no longer corresponds to the conditions on the workpiece: Although I still get -200mBar displayed here, the clamping force is reduced precisely at the most sensitive points for holding down

at the edge of the workpiece
in areas adjacent to those where the workpiece is not actively sucked in
as well as at every through-milling and a certain radius

to almost zero.

For all other areas, Kirchhoff’s laws apply: A voltage divider is formed consisting of the pump’s internal resistance, the vacuum system and the penetration resistance of the workpiece. Only the voltage still on the workpiece determines the hold-down force. It is - I repeat myself now - possibly much lower than a glance at the pressure gauge would suggest.

Solutions

Quite a lot of theory for the banal realization that the workpiece is twisted on the machine bed, isn’t it? I have the following solutions in my quiver for such a problem:

Pump / vacuum table

When working with porous materials, it is important that the table itself has a low flow resistance. This is the only way to ensure that the majority of the negative pressure falls on the workpiece, resulting in a higher clamping force. According to the motto “a lot helps a lot”, a pump with a higher flow rate, a table with larger suction openings and a vacuum distributor with larger hose diameters would be appropriate here. However, this is out of question for me.

Adhesion agent

I could try using a coarser-meshed rubber mat. With a 20mm hole pattern, the resistance for the cross air flow is tripled, so that the outer areas are no longer affected by the pressure loss to such an extent. Such a mat also has more rubber surface area, which increases static friction.

Adhesive tape instead of cling film on the material surface solves the problem of catching in the cutter and closes the pores of the material locally, which can improve the holding force in critical areas.

Of course, conventional methods such as applying double-sided adhesive tape or screwing down the workpiece can also be used on a vacuum table. But I don’t do that - because then I would have to admit to myself that my vacuum table is not suitable for all cases.

Fixture and taps

This solution aims to prevent the workpiece from twisting and shifting. The vacuum table is only used here to hold down the workpiece, while a suitably manufactured holder absorbs forces in the ‘XY direction’. The picture shows holders for the template and the gear wheel, which fulfill this task. They are attached to the machine bed and require a precise zero point in order to be able to break the workpieces edges without misalignment.

I use a clearance fit with 0.1mm between the components. If the clearance is smaller, I need too much force to insert. If I use larger values, the workpieces can be twisted too much.

With this variant, I can straighten warped workpieces, continue to produce very fine engravings and still make sure that nothing slips.

The disadvantages are time and material required for planning and producing individual fixtures or templates.

Modification of the workpiece

This is obvious: I can change the properties of the workpiece so that it becomes impermeable to air. For example, I can cover the underside with a film to improve the static friction.

If, like me, you don’t like using plastic: I have found that baking paper / cling film made from unbleached cellulose hardly lets any air through. I’ll soon be trying out covering materials or covering areas affected by milling with it instead of plastic film.

If such a problem occurs regularly, grid vacuum tables with a sealing cord are an excellent solution: Here, a linear seal is used to seal the outer edges of the workpiece against the ambient air pressure so that the edges of the workpiece have no chance of curving upwards. With my perforated grid table, however, the entire surface of the rubber mat acts as a seal, so that the curvature is in the form of a contact line on the mat. However, lines do not have a surface area, and my table can only hold down through this. Incidentally, this is also the reason why flexible materials such as films can be pulled off the vacuum table so easily. ↩︎

Maker Faire Ruhr 2024

Schallbert — Mon, 25 Mar 2024

Preparation

Since the beginning of January, two of my friends and I have been preparing for the Maker Faire Ruhr at the DASA Dortmund. In the course of this, I veneered fiberboard, cut out blanks for engravings, procured tablecloths, machined PC housings, milled gear wheels, programmed LED strips, designed a stand sign, set up a CNC machine, optimized it for engraving and so on…

As the deadline approached, things got tight. A lot of time and coordination goes into preparing for a trade faire like this.

Projects

At the faire I wanted to present some of my most beautiful projects such as Seifenbutler, my oversized spirographs and allow visitors to have their own signs engraved, even with QR code if required.

To make the milling work quicker, I made blanks that only had to be engraved later. I also had to make the spirographs in advance because of the double-sided machining required.

Machines

I was able to convince a few companies to lend me machines and parts for the faire:

From Sorotec I borrowed a “Hobbyline4530”, their smallest and lightest machine.

The company Stritzelberger provided me with a vacuum table that matches the machine, including a suitable pump.

EdingCNC supported me in setting up their software, lent me exhibitors for the latest control hardware and finally even came by my stand live for an exchange.

The stand sign

I came up with something very special for the stand sign: a planetary gear! Three moving gears revolve around a fixed sun wheel. In order to be able to suspend the stand shield and keep the gear without edges, a centered axle for the planetary carriers was out of the question.

This task is performed by a ring gear concealed by the sun gear and held by three ball bearings (image: arranged at a 120° angle, mounted on the teeth of the sun gear). It is driven by a stepper motor with a reduction ratio of 1:5. The ring gear has mounts for the planet carriers. This frees up the middle section of the sun wheel, where I can fit the drive motor, lighting and shield mounts.

A video on my PeerTube channel shows how I built the stand sign using my CNC.

Making of Standschild (Maker Faire Ruhr)↗

Delays

As always with such major undertakings, a few things went wrong during the preparations:

Detachment of the vacuum during the production of the spirographs

I wanted to make the spirographs from veneered MDF, as the material is very attractive to look at and at the same time stable enough not to be damaged by rough handling. So I covered 10mm thick medium density fiberboard with olive wood veneer (time consuming!) and then routed it on my portal router.

As I use a vacuum clamping system, it is key to achieve a certain vacuum level. Otherwise the workpieces won’t stay in place. MDF is permeable to air, so the pump has a lot to do even when the panel has not yet been processed. Unfortunately, the geometry of my design for the spirographs is such that the surface remaining for clamping continues to decrease considerably during machining.

In the end, I had the problem that the gears wanted to turn at the latest when machining the edges of the back. Even my desperate attempts to maintain sufficient negative pressure by applying large amounts of cling film to the workpiece failed.

So instead of the planned ten spirographs, I only received four - the rest were rejects. Note: MDF can only be used on my vacuum clamping system for large workpieces.

Problematic gear pairing on the spirograph

Unfortunately, the first spirograph made of MDF was not really usable: one tooth was constantly jammed on the flank of the ring gear and refused to mesh. The reason for this was that I had not provided any backlash and had manufactured the tooth heads without a radius.

As a result, the gear wheel must always be pressed firmly against the ring gear to enable the next tooth engagement. If the gear wheel slips out just a few tenths of a millimeter, it locks. Remedy: Provide both gear and ring gear tooth heads with r=3.0mm and design the gear pairing with backlash of a=0.25mm.

Incorrectly engraved lettering on the large stand plate

I made two mistakes here and unfortunately produced a lot of rejects. On the first attempt, I had overlooked to mirror the gear wheel for the rear machining. I thought this wasn’t necessary because the gear itself is mirror-symmetrical. Unfortunately, however, the retaining pins were not embedded with mirror symmetry, so that the back was not aligned with the front.

On the second attempt - see video above - I had fixed this, but forgot to adjust the zero point according to the new centering pin position on the back. Fortunately, I noticed this during processing and was able to correct it.

Retro-PC window production

On one of the three retro PCs, I placed the back of the housing on the machine bed rotated by 180°. The window was therefore milled “upside down”, which I only noticed after finishing the milling job. Fortunately, my friend was able to file and bend the latches so that the back of the housing could also be attached to the PC “upside down”.

During the production of the last window, the chip guard got caught in the workpiece’s splay and broke because I couldn’t get the machine to stop in time. This could only happen because the chip guard became loose due to the vibrations during machining. Note: The milling machine is not a toy. So don’t make any compromises or use half-baked makeshift solutions!

The Maker Faire Ruhr 2024

The fair itself was great. I received a very friendly welcome and DASA gave me excellent support with unloading and all other logistics. The stand assigned to me was also great: very large, well located, with plenty of power sockets - what more could you want?

I had a company visit my stand, family and friends came and even some people from the CNC forums and a few colleagues from work came to see me. I couldn’t have wished for more.

Set-up

Fortunately, we had a forklift to help us unload.

Your browser does not support the video tag.

Unloading the CNC

Everything else was manual work and not much different from setting up for a music concert: Moving tables, laying power cables, installing the lighting system, setting up exhibits.

Impressionen von der Maker Faire Ruhr 2024.

Hier wird der Vakuumtisch in Betrieb genommen und getestet.

Die CNC-Maschine für die Messe. Fertig eingerichtet und betriebsbereit.

Selbstgefräster Schilder für die dargestellten Projekte.

Das Standschild für einen guten Freund, der Retro-PCs und -Spiele ausstellt.

Schöner Abfall - ein Reststück Acrylglas in Form einer Welle

Das Gegenstück zur Welle: Ein PC mit einzigartigem Fenster, selbstgefräst.

Die LED-Streifen sind fertig programmiert und alles steht an seinem Platz.

Retro-2000er-PCs sind aufgebaut und vernetzt. Bereit zum Spielen.

Die betriebsbereite CNC-Maschine auf der Messe.

Bereits einige Besucher, Maker und Interessierte am Samstag.

Backstage des betriebsbereiten Standes.

Doch am Sonntag war noch viel mehr los.

Visitors

I can draw a positive balance for my first Maker Faire: There were really a lot of people at our stand and I hardly had a few minutes to rest - there was always something to talk about, to present my projects or to talk about the machine on display. But that was exactly what attracted us to this format. What’s more, the technology worked, the machine ran smoothly and all the things I had prepared so long in advance (such as the stand sign) could be exhibited or shown in operation.

Your browser does not support the video tag.

QR-Codengrave Demonstration

What I would still do differently next time

The Maker Faire is not a sales fair in terms of setup, but a huge show & tell. That’s why I wouldn’t book a paid sales stand again, because in my case it wasn’t worth it.
One or two more people to look after the stand would be great. Then we could take turns.
More material for the spirographs! All of the pens we brought were broken by the end of the event and had run out of paper by the middle of the second day.
More presentations / demos. The milling machine was often running, but there was a lack of templates and requests from the visitors. Next time I will take more of my own designs with me and produce them for demo purposes even without a request.
Prepare better links to the website, because a QR code alone doesn’t help everyone.
More coffee: The supply of hot drinks (and now that I write it, food too) was undersized and unfortunately not as tasty as I would have liked. So it’s better to make own sandwiches and bring own thermos flasks.

Seifenbutler - Pilot run!

Schallbert — Thu, 08 Feb 2024

My Hardware project Seifenbutler is complete. After more than a year of development, I have manufactured some in white, black, transparent, and white translucent. They even made it to a first online shop.

How do I determine the sales price?

As I use high-quality materials and the production and assembly time is long, I don’t have much room for maneuver when it comes to pricing. On the other hand, I can’t ask moon prices for a commodity like Seifenbutler - unless I use precious woods or other unusual or complex materials. There is a lot of material online for such calculations. I used a YouTube video from the “Cutting It Close” channel as a guide.

My price calculation is made up of the following items:

Final price = material price
 + machine time (investment, wear, tools, consumables)
 + production and assembly time 
 + packaging costs 
 + fees (postage, payment service providers, stores)

As you can see, I have not included my own time for product development in the calculation - if I had to do this, it would easily add the equivalent of 250h of working time.

Material prices

As of the end of 2023

Ok, let’s quickly list the expenses per part:

quantity	item	price/unit	unit used	price
2	PMMA 5mm XT	90€/m²	0,03	2,70€
1	Knurl M4 A2 22mm	7,54€/25pcs	0,04	0,30€
1	Washer M4 A2 12mm	0,94€/100pcs	0,01	0,01€
1	pin M4 A2 12mm	6,70€/50pcs	0,02	0,13€
5	screw M4 A2 8mm	12,40€/200pcs	0,025	0,31€
			Material cost/part	3,45€

Machine time

I estimate the machine time for this project at 100€/h. The lion’s share of this is the investment, maintenance and wear costs of the machine at €50/h plus my own working time, which I simply estimate at €30/h.

The tool costs are quite high at around €15/h, as I have to use four different tools, all of which are small in diameter and therefore subject to high wear.

Consumables are vacuum fleeces, positioning pins, cut protection gloves and adhesive tape. I need the latter to fix the small milled parts and the spiral. Their share of the machine time costs is around 5€/h.

The machine time is currently 8min/part. This means that the item machine time amounts to 13.30€/piece.

Assembly time

It takes me about 5 minutes to assemble a soap holder. If I apply an hourly rate of 30€, I arrive at 2.50€ per item.

Packaging costs and purchase of materials

I pack the soap holders in simple DIN A6 envelopes after wrapping them carefully in wrapping paper. The costs here amount to just 0.20€ per item. However, if I now include my working time for packaging and shipping labels, we are already at 0.90€ per item. Added to this are the costs for postage when purchasing materials and the costs for paper and printing the operating instructions, which I calculate at a further 0.50€/unit - the small quantities produced are weighing heavy on this.

Fees

Fees are incurred for shipping and payment service providers, for example. Here the prices vary depending on the provider, so I plan a shipping surcharge of 3.50€ (shipping by large letter) - this does not apply to sales “at the counter”.

The total price

If I add up all the items, I would have to call up a store price of 20.15€ (cash) for the unpacked goods and 24.75€ in the case of shipping with cashless payment. And then the company would still not have made a cent of profit.

I’m not sure whether the soap butler can be sold for such prices. But if you’re honest and don’t want to end up paying more, such calculations are unavoidable, at least roughly.

The assumed merchant with a store

If I were a merchant¹, I would still have to add VAT and would also want to see the development costs amortized somehow. I’m also not sure whether I would get through the door “as a merchant” with an hourly rate of 30€.

Phew. If you now include things like store rent, costs for storage space, workshop rent, electricity, waste disposal, membership of the “Green Dot” in accordance with the Packaging Act, contributions to the Chamber of Industry and Commerce, the employers’ liability insurance association as well as taxes and duties, then even with a further 20% mark-up, a whole heap of soap holders would have to be sold in order not to end up poorer than before!

Let’s assume I can rent a workshop and open a tiny store.

A fictitious store

Let’s do a quick calculation with a few assumed values.

Item	Price/Unit	Used	Final price
Shop	16€/m² per month	40m²	640€
Storage space	4€/m² per month	8m²	32€
workshop	8€/m² per month	30m²	240€
Service charges	per month	1	200€
Taxes	IHK / BG / insurance	1	125€
Fees	20€/month	3	60€
		Total	1297€
		Staff	1600€

And if I were to hire someone to work in this store for 20hours/week, I would incur an estimated €1600 in costs.

Summa summarum

Ok, let’s summarize: I start with a mark-up of 20% on the production costs. As an assumed merchant, however, I would have to pay VAT, which I would have to add on again.

Retail price = production price 
 + contribution margin (20%)
 + value added tax (19%)
 = (1,2 * 20,15€) * 1,19 = 28,77€

Oh well, that puts us at almost thirty euros per item.

So how many soap butlers have to be sold per month before I would make a profit as a merchant with a store? The contribution margin is 4.03€ (20% of the production price). If I now divide the costs generated by production and sales by this amount, I get my minimum sales quantity: 719 units/month.

So how many hours per week would I have to spend at least at my CNC milling machine or assembling and packaging soap butlers? If I take the values from above, I get a total of 8 + 5 + 2 = 15min/part.

With the minimum sales volume, that would give me a working week of about 45 hours! Oh, 60 hours a week is easy, isn’t it? So I could produce 960 units per month - and assuming they sell excellently, I would earn an extra €971 to amortize the development costs (estimate 250h*€30 = €7,500). That’s a bang!

I would then march into the profit zone with the company after just eight months (with 60 hours per week, of course). Oh, or do you still have to pay taxes? And when was my machine paid off?

Conclusion on profitability

I realize that the calculation is not that simple. Some parameters cannot be scaled up or down arbitrarily and linearly and I cannot realistically estimate the machine costs, for example, because I simply don’t have any experience in continuous operation - keyword wear and tear.

Nevertheless, the amount of work I have to do is enough for me to say:

It won’t work. Nice project, but not worth it, at least not with my own funds and on my own initiative. It remains a pilot series and my joy at a successful, functional product.

Fortunately, this is all “just my hobby”, so I’m not forced to draw the disastrous conclusions 😉

Careful. My vocation is not to be a trader or a businessman, but to be an engineer. That’s why I only have a very rudimentary knowledge of business administration (just that one semester at university when nobody really wanted to listen). All my comments here are rough estimates. They should therefore be taken with a grain of salt. However, I am very happy to receive corrections and constructive criticism! ↩︎

'redir' through Caddyserver

Schallbert — Tue, 30 Jan 2024

When I moved my website from Github Pages to my own server, I had some problems and unanswered questions regarding automatic redirects. My blog migrated from the root domain to a subdomain (blog.schallbert.de).

Requirements for the redirect

Links already referencing to my blog articles under the old name, e.g. to schallbert.de/about should redirect correctly to the subdomain (no 404)
The root page under schallbert.de should be fully usable
Data and files for domain and subdomains should be independent of each other
The web servers should be able to use different technologies for different subpages

Implementation

At first I had thought that I could achieve the redirects in the DNS provider via A and CNAME entries. However, I quickly realized that a) I had no idea how DNS even works and b) the right way is via the configuration of my web server.

First attempt: redir at ‘/’

On my web server (Caddy), redirects are specified in the Caddyfile via redir. This can be done permanently (as so-called http 301) or temporarily via 302. Default is a temporary redirect.

My first thought was to leave schallbert.de unmodified and forward everything after a possible /. This would then look like this in the Caddyfile:

schallbert.de {
 # Define webserver
 root * /www/landing
 encode gzip
 file_server

 # Redir to subdomain if domain has a slash
 @redirect path_regexp /*
 redir @redirect https://blog.schallbert.de{uri}
}

As a result, Caddy simply forwarded everything to the subdomain. The catch was that the subdomain itself also contains slashes or a URI after the slash and is then forwarded yet again. Example:

schallbert.de/post -> blog.schallbert.de/post -> blog.schallbert.de/post -> blog.schallbert.de/post -> [...]

This gave me a recursive endless redirect. I completely paralyzed my server with it 😅

Second attempt: Filtering via regexp

I need better filtering so that only slashes with additional characters behind them are actually redirected and the forwarding cannot be recursive. Fortunately, I had help from a good friend, so that the following Caddyfile was created a short time later:

schallbert.de {
 # Define webserver
 root * /www/landing
 encode gzip
 file_server

 # Redir to subdomain to maintain links for blog
 @redirect path_regexp ^/[^/]+(/.*)?$
 redir @redirect https://blog.schallbert.de{uri}
}

The Regexp checks whether there is a slash after the domain URL and whether it is followed by at least one other character. The redirection now works as desired and my blog is still fully accessible under the old links.

Nevertheless, I was still not completely satisfied. Because when I created a nice “landing page” on my root domain, suddenly only the plain HTML was visible. No favicon, no images, no CSS.

A quick look at the developer options quickly brought the realization that paths of all assets that I had stored on my landing page, marked with / in the folder structure, were forwarded to the subdomain. Where they are not located, of course.

Third attempt: Regexp plus file search

Fortunately, I’m not alone with this problem, so I found the solution in Caddy’s forum:

With the not file marker I can define redirects in Caddyfile if resources on the current page cannot be found.

My solution now looks like this:

schallbert.de {
 @filenotfound {
 path_regexp ^/[^/]+(/.*)?$
 not file
 }

 # show Caddy where to find page resources
 root * /www/landing
 encode gzip

 route {
 # Redirect to subdomain if article cannot found on root
 redir @filenotfound https://blog.schallbert.de{uri}

 # else, define webserver and show page
 file_server
 }
}

Here it is important to define the page with root * /www/<pagelocation> before creating the route. Otherwise the assets will only become available after I have already forwarded them for lack of existing files.

Success!

Now my construction site page is finally displayed correctly.

Automatic Upgrades for everything!

Schallbert — Sat, 20 Jan 2024

As the title says, I would like to have security updates installed as soon as they are released. This applies to the Ubuntu on my cloud server as well as to all applications that I run in Docker. As I’m lazy, I don’t want to carry out these security updates manually.

For function updates and enhancements, however, it is still OK for me to intervene manually from time to time. Let’s start with the system upgrades.

Security updates per `unattended-upgrades`

Ubuntu supplies - as a short query reveals - a configurable tool for automated upgrades called unattended-upgrades. First, I check whether it is installed on my system:

Installation

schallbert:~# which unattended-upgrades
/usr/bin/unattended-upgrades

Great. Otherwise I’d install it with apt install unattended-upgrades.

configure unattended-upgrades

The Debian Wiki to the rescue: Config files can be found at /etc/apt/apt.conf.d.

First I make sure that automatic upgrades are active. To do that I open 20auto-upgrades and check presence of following lines:

APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Unattended-Upgrade "1";

Very well, so automatic package index updates and -upgrades are active. Now, I should set an update time and upgrade types to install. These can be configured within50unattended-upgrades, mostly by removing commented-out lines.

Upgrade sources

The first section of the configuration file is dedicated to the permitted sources for the updates. Of course, only trustworthy providers should be listed here. My configuration includes updates for my Linux distribution and security plus ESM = “Enhanced Security Maintenance” updates. It looks like this:

Unattended-Upgrade::Allowed-Origins {
 "${distro_id}:${distro_codename}";
 "${distro_id}:${distro_codename}-security";
 "${distro_id}ESMApps:${distro_codename}-apps-security";
 "${distro_id}ESM:${distro_codename}-infra-security";
};

Updates for dev releases, too?

I have the value set to auto:

Unattended-Upgrade::DevRelease "auto";

Automatic reboot

Some security updates (e.g. concerning the kernel) require the machine to be restarted. They will therefore only take effect if you have set the corresponding values in the file. WithUsers is a matter of taste, because you are force-logged off if by chance an update has just been installed that requires a restart. To practically rule this out, I have entered a time under Reboot-Time when I am certainly not working with the machine.

Unattended-Upgrade::Automatic-Reboot "true";
Unattended-Upgrade::Automatic-Reboot-WithUsers "true";
Unattended-Upgrade::Automatic-Reboot-Time "03:45";

Of course, automatic reboot has repercussions for all the programs I use. I therefore make sure in my docker-compose.yml files that always or unless-stopped is really noted in restart:.

Notifications

I haven’t switched on automatic mails or other push notifications for the time being, as I’m still in test mode and don’t really want to receive a lot more messages. As soon as I change this, there will definitely be an update here.

Function test

To see if unattended-upgrades does what it is supposed to do, I take a look at the logs under /var/log/unattended-upgrades. The file unattended-upgrades.log contains the following, for example:

# [...]
2024-01-12 06:17:59,781 INFO Starting unattended upgrades script
2024-01-12 06:17:59,781 INFO Allowed origins are: o=Ubuntu,a=jammy, o=Ubuntu,a=jammy-security, o=UbuntuESMApps,a=jammy-apps-security, o>
2024-01-12 06:17:59,781 INFO Initial blacklist:
2024-01-12 06:17:59,782 INFO Initial whitelist (not strict):
2024-01-12 06:14:33,956 INFO Packages that will be upgraded: libc-bin libc-dev-bin libc-devtools libc6 libc6-dev locales python3-twisted
2024-01-12 06:14:33,956 INFO Writing dpkg log to /var/log/unattended-upgrades/unattended-upgrades-dpkg.log
2024-01-12 06:14:54,578 INFO All upgrades installed
# [...]

However, the file unattended-upgrades-shutdown.log is still empty. I will check at a later date whether everything works here too.

Automate container updates with `Watchtower`

Now to the updates of my applications. A quick chat with a couple of admins revealed this:

It is tedious to keep all programs up to date manually
For containerized applications, there are services that solve this task centrally
There have already been bad experiences when the latest release is referenced in docker-compose.yml.
With image: <application>:latest, not all applications are stable or create problems with dependencies

One of these solutions is offered by Watchtower, which again runs in a container.

Installation

As usual I create a docker-compose.yml in a new folder at opt/watchtower:

# copied from https://containrrr.dev/watchtower/notifications/

# watchtower/docker-compose.yml
version: "3"
services:
 watchtower:
 image: containrrr/watchtower
 restart: unless-stopped
 volumes:
 - /var/run/docker.sock:/var/run/docker.sock

Watchtower needs docker.sock to pull the updates and apply them to the existing containers. Apart from that, I have done nothing else for the time being. No notifications (for the reasons mentioned above), no logs, nothing. I hope this setup stays functional and I don’t have to post an update here anytime soon.

Further steps for a validation of Watchtower’s functions, for example, can be found in a tutorial by DigitalOcean.

Procedure

Watchtower checks the installed Docker images for updates on a daily basis according to the documentation. If there is a new image release, Watchtower sends a SIGTERM signal to the containers to be updated, whereupon they shut down (source). The containers are then started up again.

Involuntary functional test

A few days after starting the container, I dialed into my server again and looked through some logs at random. I just wanted to check what was going on.

In the fail2ban log files I suddenly saw pages and pages of entries at a certain point in time, 14:42:56. More than I had seen in sum for days before. When I scrolled up to the beginning of this chain of entries, I saw that fail2ban had probably stopped for a moment.

 2024-01-26 14:42:50,817 <container_id> INFO Shutdown in progress...
 2024-01-26 14:42:50,817 <container_id> INFO Observer stop ... try to end queue 5 seconds
 2024-01-26 14:42:50,837 <daemon_id> INFO Observer stopped, 0 events remaining.
 2024-01-26 14:42:50,878 <container_id> INFO Stopping all jails
 2024-01-26 14:42:50,879 <container_id> INFO Removed logfile: '/var/log/auth.log'
 2024-01-26 14:42:50,879 <container_id> INFO Removed logfile: '/var/log/caddy2/gitea_access.log'
 2024-01-26 14:42:50,879 <container_id> INFO Removed logfile: '/var/log/caddy2/server_access.log'
 2024-01-26 14:42:51,052 <daemon_id> NOTIC [sshd] Flush ticket(s) with iptables
 2024-01-26 14:42:51,063 <container_id> INFO Jail 'sshd' stopped
 2024-01-26 14:42:51,137 <daemon_id> NOTIC [caddy-status] Flush ticket(s) with iptables-multiport
 2024-01-26 14:42:51,137 <container_id> INFO Jail 'caddy-status' stopped
 2024-01-26 14:42:51,138 <container_id> INFO Connection to database closed.
 2024-01-26 14:42:51,139 <container_id> INFO Exiting Fail2ban
 2024-01-26 14:42:56,173 <new_container_id> INFO --------------------------------------------------
 2024-01-26 14:42:56,173 <new_container_id> INFO Starting Fail2ban v1.0.2
 2024-01-26 14:42:56,173 <new_container_id> INFO Observer start...
[...]

All the log entries at the same time start at the end of the section shown above. A whole bunch of IP addresses are loaded there in preparation to be banned. I looked at the Ubuntu syslog, slightly worried:

Jan 26 14:42:55 schallbert systemd[1]: docker-<ID>.scope: Deactivated successful
Jan 26 14:42:55 schallbert systemd[1]: docker-<ID>.scope: Consumed 21:05 CPU time
Jan 26 14:42:55 schallbert dockerd[690]: time="2024-01-26T14:42:55.228607617Z" level=info msg="ignoring event" 
Jan 26 14:42:55 schallbert containerd[639]: time="2024-01-26T14:42:55.229864632Z" level=info msg="shim disconnected" 
Jan 26 14:42:55 schallbert containerd[639]: time="2024-01-26T14:42:55.230125409Z" level=warning msg="cleaning up after shim disconnected" i>
Jan 26 14:42:55 schallbert containerd[639]: time="2024-01-26T14:42:55.230145478Z" level=info msg="cleaning up dead shim"
Jan 26 14:42:55 schallbert containerd[639]: time="2024-01-26T14:42:55.240586557Z" level=warning msg="cleanup warnings time=\"2024-01-26T14:>
Jan 26 14:42:55 schallbert dockerd[690]: time="2024-01-26T14:42:55.241839655Z" level=warning msg="ShouldRestart failed
Jan 26 14:42:55 schallbert containerd[639]: time="2024-01-26T14:42:55.870912387Z" level=info msg="loading plugin \"io.containerd.event.v1.p>
Jan 26 14:42:55 schallbert containerd[639]: time="2024-01-26T14:42:55.870997689Z" level=info msg="loading plugin \"io.containerd.internal.v>
Jan 26 14:42:55 schallbert containerd[639]: time="2024-01-26T14:42:55.871007368Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.t>
Jan 26 14:42:55 schallbert containerd[639]: time="2024-01-26T14:42:55.871150831Z" level=info msg="starting signal loop" namespace=moby path>

Hm, cleaning up dead shim. That sounds ominous. A quick search shows that the shim forms an intermediate layer between the container manager and the container itself and forwards the container’s input and output. Its runtime is linked to that of the container.

In other words, I saw a normal shutdown followed by a restart. Fortunately, no break-in! All that remains is to find out the reason for the restart of Fail2ban. This reminds me that I don’t even know when Watchtower runs the updates…

So I quickly looked in the Watchtower logs with docker container logs <container_id> and behold:

time="2024-01-26T14:42:47Z" level=info msg="Found new lscr.io/linuxserver/fail2ban:latest image (9bda077d765e)"
time="2024-01-26T14:42:50Z" level=info msg="Stopping /fail2ban (e0317a105d53) with SIGTERM"
time="2024-01-26T14:42:55Z" level=info msg="Creating /fail2ban"
time="2024-01-26T14:42:55Z" level=info msg="Session done" Failed=0 Scanned=6 Updated=1 notify=no

Now I have proof: Watchtower does what it is supposed to do. It upgrades containers to a new version as soon as updates become available. Great!

PS: Since I haven’t configured Watchtower very much, I assume from the timestamps in the logs that the updates are run daily at the time Watchtower is started. I wonder whether I should switch to a certain version of the image instead of using latest, though.

Better security for my server

Schallbert — Fri, 12 Jan 2024

Motivation

OK. I have my server setup, created an automatic build pipeline for my Blog and my website is displaying the Blog as it should. Still, I’m not completely done yet. Because I want to do more for server-side security than deactivating password logins.

And I don’t have backups. That’s never good, so let’s get something done about that.

Locking away unwanted guests

In my SSH-log of my machine’s root system, I get a lot of failed authentications:

[...]
Jan 11 00:19:29 sshd[93842]: Invalid user admin from 41.207.248.204 port 37194
Jan 11 00:19:29 sshd[93842]: pam_unix(sshd:auth): check pass; user unknown
Jan 11 00:19:29 sshd[93842]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ru>
Jan 11 00:19:31 sshd[93842]: Failed password for invalid user admin from 41.207.248.204 port 37194 ssh2
Jan 11 00:19:32 sshd[93842]: Connection closed by invalid user admin 41.207.248.204 port 37194 [preauth]
Jan 11 00:20:14 sshd[93886]: Invalid user svn from 84.108.40.27 port 44968
Jan 11 00:20:14 sshd[93886]: pam_unix(sshd:auth): check pass; user unknown
Jan 11 00:20:14 sshd[93886]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ru>
Jan 11 00:20:16 sshd[93886]: Failed password for invalid user svn from 84.108.40.27 port 44968 ssh2
Jan 11 00:20:16 sshd[93886]: Received disconnect from 84.108.40.27 port 44968:11: Bye Bye [preauth]
Jan 11 00:20:16 sshd[93886]: Disconnected from invalid user svn 84.108.40.27 port 44968 [preauth]
[...]

If I didn’t know that this is typical “internet noise”, I’d be nervous. Isn’t it coming close to thieves trying different keys at our front doors every few seconds? So what can we do about it? Ban them.

fail2ban

That’s exactly what fail2ban can do for me.

 __ _ _ ___ _ 
 / _|__ _(_) |_ ) |__ __ _ _ _ 
 | _/ _` | | |/ /| '_ \/ _` | ' \ 
 |_| \__,_|_|_/___|_.__/\__,_|_||_|
 v1.1.0.dev1 20??/??/??

In short, fail2ban scans access/auth logs¹ for IP-addresses causing multiple failed authentications, and bans them when surpassing a user-defined threshold within a defined period of time.

How does fail2ban work? It modifies iptables, thus accesses packet filter rules (buzzword Firewall) on the network layer. So incoming requests of already blocked IP-addresses will not even reach my applications².

install fail2ban

fail2ban seems to be close to an industry standard for blocking unwanted access requests on Linux. Every hobbyist admin I know is using it.

For installation and configuration of fail2ban there’s a ton of guides out there plus the (well-written) on inside its Github-repository. I won’t be pressing the point here.

I chose an installation as Docker container to get all dependencies auto-delivered as well. I’m using the distribution by linuxserver and write the following docker-compose.yml to have it deployed:

# /fail2ban/docker-compose.yml
version: "2.1"
services:
 fail2ban:
 image: lscr.io/linuxserver/fail2ban:latest
 container_name: fail2ban
 cap_add:
 - NET_ADMIN
 - NET_RAW
 network_mode: host
 environment:
 - PUID=1000
 - PGID=1000
 - TZ=Etc/UTC
 - VERBOSITY=-vv #optional
 volumes:
 - ./config:/config
 - /var/log/auth.log:/var/log/auth.log:ro # host ssh
 - /var/log/caddy2:/var/log/caddy2:ro  # gitea via caddy, caddyserver
 restart: unless-stopped

What I add to add here: fail2ban requires the Access logs as Volume (above added with :ro as read-only volumes). A lot of filter rules come predefined in its config folder, I just had to add a modified jail.local inspiration: linuxserver/fail2ban-confs and add a rule in filter.d for caddy pulled from muetsch.io to complete my door watch.

fail2ban example

This is how a fail2ban-log looks like for my SSH-Daemon (sshd):

# schallbert:/opt/fail2ban/config/log/fail2ban# grep "220.124.89.47" fail2ban.log 
 2024-01-11 17:12:20,600 7FBB3630BB38 INFO [sshd] Found 220.124.89.47 - 2024-01-11 17:12:20
 2024-01-11 17:12:23,003 7FBB3630BB38 INFO [sshd] Found 220.124.89.47 - 2024-01-11 17:12:22
 2024-01-11 17:12:25,205 7FBB3630BB38 INFO [sshd] Found 220.124.89.47 - 2024-01-11 17:12:24
 2024-01-11 17:12:27,206 7FBB3630BB38 INFO [sshd] Found 220.124.89.47 - 2024-01-11 17:12:26
 2024-01-11 17:12:32,610 7FBB3630BB38 INFO [sshd] Found 220.124.89.47 - 2024-01-11 17:12:32
 2024-01-11 17:12:33,045 7FBB36104B38 NOTIC [sshd] Ban 220.124.89.47

kbye!

What does not (yet) work: Gitea & fail2ban

Also on my gitea instance, I get quite some authentication requests that I’d like to block. Unfortunately, I cannot get Gitea to put the registered accesses into a log file. I see them in the console only. On the other hand, the shell access to Gitea is already secured as it is channeled through sshd that also guarantees remote access to my server.

What is missing is a ban for failed web login requests.

I thought I had configured app.ini to get access logs written to file:

# gitea/conf/app.ini
#[...]
[log]
MODE = file
LEVEL = warn
ROOT_PATH = /data/gitea/log
ENABLE_ACCESS_LOGS = true
ENABLE_SSH_LOG = true
logger.access.MODE = access-file

[log.access-file]
MODE = file
ACCESS = file
LEVEL = info
FILE_NAME = access.log
[...]

I have both activated the logs with ENABLE_ACCESS_LOGS and configured log-to-file. access.log is being generated but no authentication tries are being written to it. Maybe caddy’s reverse proxy snitches them away before they reach Gitea? I’m sure I’ll find out at a later point in time.

At some point I asked myself whether fail2ban actually works on the iptables of my server or only operates within the container. Luckily, I was not the first with this question (stackoverflow) and found the proposed solution quite charming:

Block any IP in the Docker container (docker exec -it fail2ban sh): fail2ban-client set sshd banip 111.111.111.111
Check the iptables to see if this IP appears there: iptables -n -L --line-numbers
In my case: enjoy, because it is there: 1 REJECT all -- 111.111.111.111
Unblock the IP again fail2ban-client set sshd unbanip 111.111.111.111

Protection against brute-force attacks via ssh as described above
Protection against overload using a Rate Limiter
Protection against attacks on the APIs (404/403 attacks) using caddy-status configuration for fail2ban, logs in JSON

# /config/fail2ban/filter.d/caddy-status.conf

# this regex works for caddy with json-style logs
failregex = "client_ip":"<HOST>"(.*)"status":(400|401|403|404|500)
datepattern = \d+
ignoreregex =

Regular backups

I want to be able to do disaster recovery. Unforeseen events impacting my server like incompatible updates, machine downtime or security breaches should not keep me from spawning and configuring a new, healthy instance in no time.

I’m not willing to create backups manually, copy and compress folders, finally downloading to a backup location via SCP / SFTP. This should take place automatically, and, ideally, without cost. A quick search reveals two suitable open-source tools: borgmatic and restic.

I randomly chose Borgmatic.

install Borgmatic with Docker

As for all other components so far, I want Borgmatic to run in Docker environment. Luckily there are ready-made solutions that need just a little configuration.

This is how my docker-compose file looks like:

# /borgmatic/docker-compose.yml
version: '3'
services:
 borgmatic:
 image: ghcr.io/borgmatic-collective/borgmatic
 container_name: borgmatic
 volumes:
 - ${VOLUME_SOURCE}:/mnt/source:ro  # backup source
 - ${VOLUME_TARGET}:/mnt/repository  # backup target
 - ${VOLUME_ETC_BORGMATIC}:/etc/borgmatic.d/  # borgmatic config file(s) + crontab.txt
 - ${VOLUME_BORG_CONFIG}:/root/.config/borg  # config and keyfiles
 - ${VOLUME_SSH}:/root/.ssh  # ssh key for remote repositories
 - ${VOLUME_BORG_CACHE}:/root/.cache/borg  # checksums used for deduplication
 # - /var/run/docker.sock:/var/run/docker.sock # add docker sock so borgmatic can start/stop containers to be backupped
 environment:
 - TZ=${TZ}
 - BORG_PASSPHRASE=${BORG_PASSPHRASE}
 restart: always

Inspirations were the related docs on Github. You can see that Borgmatic requires a lot of volumes to work properly, most importantly backup source and target location.

configure Borgmatic

All concrete data behind ${} are summarized in an .env environment variable file. This helps me avoid publishing passphrases by accident. Of course I have noted down the passphrase on paper as “backup” of the backup.

I then modify config.yml in borgmatic.d/ slightly, adding backup source, target, count, and cycle. That’s all, I’m ready for a first test.

testing Borgmatic backup

docker exec borgmatic bash -c \
"cd && borgmatic --stats -v 1 --files 2>&1"

This command has borgmatic create a backup for me through the Docker container.

I got an error message right away, saying repository does not exist. A look into the docs reveals that I have to manually create the backup target repository.

The following command does this, creating an empty, encrypted repo:

docker exec borgmatic bash -c \
"borgmatic init --encryption repokey-blake2"

Now I retry manual backup creation which throws no error .

Ein Backup only is a Backup…

…when it is successfully applied, a friend of mine said.

Right he is. Still, I don’t dare replacing my working server configuration with a backup. As a compromise, I go halfway by creating a docker-compose.restore.yml following the online guide. For additional help, I read modem7’s related blog post.

For my test purposes, I run docker-compose.restore.yml in the container’s shell and enter the following commands:

mkdir backuprestoremount
borg mount /mnt/repository /backuprestoremount

mkdir backuprestore
borgmatic extract --archive latest --destination /backuprestore

Here, I create the folder /backuprestoremount and have it point to my borg backup. Then, I extract the backup into /backuprestore.

Let’s do a sanity check:

# cd /mnt/backuprestore
# /restore/mnt/source ls
borgmatic caddy2 containerd fail2ban gitea hostedtoolcache watchtower

Yeah! The backup contains all applications along with their files. I stop at this point as I’m too lazy and cowardly to overwrite my working setup as mentioned above.

What (yet) doesn’t work: Borgmatic & docker-compose down

Borgmatic offers a simple method to run pre- and post-backup actions. As I want to maintain consistency within my containers, I want them to be shutdown before and restarted after the backup run. So I wrote a script, hooked it into the config and… nothing.

Easy to follow, this can only work directly if Borgmatic ran on bare metal. Docker’s encapsulation doesn’t let borgmatic run shell scripts outside its container. I thought I could circumvent this problem by including var/run/docker.sock in the docker-compose file but without success.

I’m sure this can be solved somehow. For the time being, I have more important problems to solve and thus I’ll live with the assumption “there are no data inconsistencies³.

The SSH-Daemon of my server puts its authentication logs to /var/log/auth.log, but could also be named access.log or similar in other applications. Services like caddy or gitea also create auth log files. ↩︎
However, what I yet do not understand is how fail2ban is able to access the iptables from within the container. I thought a positive side effect of containerization is encapsulation. Or is that only complete with “rootless”-Containers? ↩︎
Background: I’m currently the only one contributing to my gitea instance. My website is static. The webserver caddy only handles changing files once pushed to Gitea. I have scheduled application updates per watchtower and unattended-upgrades (see my follow-up post) at night so they won’t interfere with borgmatic runs. Just the logs for fail2ban are still written independently. I accept the risk of data loss here because packet filtering is subject to change anyways. ↩︎

Jekyll build: broken relative links

Schallbert — Fri, 29 Dec 2023

Problem

Now, after eternal trial and error, I am finally able to provide my Caddy server with the build files via Docker-Volume. Unfortunately, the page looks like this:

Analysis

At first I suspected the web server. But when I call up the website in my browser¹ as a test, I see the same behavior. Strange.

Obviously all stylesheets are missing. But if you look closely, images and other media have not been loaded either. Only their descriptions. Then I click wildly on a few links to articles and subpages. I notice that the paths look different than I would have expected:

# --- Expectation ---
# Image
/assets/images/test.jpg
# Post
/this-is-a-post

Instead I see:

# --- Observation ---
# Image
/pages/<username>/<repositoryname>/assets/images/test.jpg
# Post
/pages/<username>/<repositoryname>/this-is-a-post

This is also shown by the browser’s developer options activated with F12 (see image above): all sources integrated via a relative link cannot be loaded. It is generally a clear recommendation to have the developer options activated in the event of problems with the display of websites.

It is somehow obvious that my Jekyll configuration file _config.yml or my translation header i10.yml contain errors. Fortunately, there are already excellent help pages.

However, a test quickly shows that I cannot leave the attribute repository empty, where the build <username>/<repositoryname> emanates from:

# bundle exec jekyll serve
Generating... 
 Jekyll Feed: Generating feed for posts
 Liquid Exception: No repo name found. 
 Specify using PAGES_REPO_NWO environment variables, 'repository' in your configuration, 
 or set up an 'origin' git remote pointing to your github.com repository. in /_layouts/default.html
 ERROR: YOUR SITE COULD NOT BE BUILT

And, heck, where does the /pages suddenly come from?

Solution

As /pages is added to paths I suspect Jekyll build options and Github-Pages Gem to be the culprits.

Of course I’m not alone with this problem, so I found the solution documented.

I just change JEKYLL_ENV (build environment) in the action script from production to development, like it is normally done for building on a local machine.

# workflows/jekyll-build-action.yml
env:
 JEKYLL_ENV: development # I had "production" here before

And what can I say: as soon as you do it right, it works!

Displaying the website offline works as follows: Drag the built files (the _site folder) to your own computer and open the top-level index.html with the browser. ↩︎

🗸 Gitea Actions Part2 - Jekyll-Dockerimage

Schallbert — Sat, 09 Dec 2023

What happened so far

I wasn’t able to set up the CI/CD pipeline in Part1 of this sequel. I tried with a minor modified copy of the Github-Action to build Jekyll for Github Pages. Now I wanted to try running Jekyll as a Dockerimage right from the start. There wasn’t too much choice on the Docker hub, still I found something which seemed well-documented and actively maintained at year-end 2023.

“Set up job”

So I add the Docker-jekyllimag to my workflow:

# workflows/jekyll-build-action.yml
jobs:
 # Build job
 build:
 runs-on: ubuntu-latest # this is the "label" the runner will use and map to docker target OS
 container: jvconseil/jekyll-docker
# [...]

Here I’m telling gitea act_runner to run with ubuntu-latest label which in my case points to a minimalistic node16:bullseye machine (Debian11). I’m then telling docker to load the jekyll-docker image, where the dependencies I require for my build are already loaded.

bundle install

Well, initial build fails due to ruby dependencies that are specified in my project’s Gemfile which are not present yet. The error message looks like this:

bundler: failed to load command: jekyll (/usr/gem/bin/jekyll)
/usr/local/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/resolver.rb:332:in `raise_not_found!': Could not find gem 'github-pages' in locally installed gems. (Bundler::GemNotFound)
	from /usr/local/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/resolver.rb:392:in `block in prepare_dependencies'

Bundler is Ruby’s packet manager. Similar to npm for Javascript or pip für Python, bundler is able to load and bind dependencies and libraries for applications running in Ruby. The Gemfile specifies the packages for bundler to install.

So let’s add bundle install to the script like so:

# workflows/jekyll-build-action.yml
# [...]
 steps:
 - name: --- CHECKOUT ---
 uses: actions/checkout@v3
 - name: --- INSTALL GEMS ---
 run: bundle install # will fail with permissions rights to write to Gemfile.lock but anyways installs required dependencies.
 - name: --- BUILD WITH JEKYLL ---
 # Outputs to the './_site' directory by default
 run: bundle exec jekyll build --destination /opt/blog_staging
 env:
 JEKYLL_ENV: production

act_runner: no write permissions to `Gemfile.lock`

This fails again. act_runner:

gitea-runner-1 | [Deploy Jekyll site/build] | `/workspace/schallbert/blog/Gemfile.lock`. It is likely that you need to grant
gitea-runner-1 | [Deploy Jekyll site/build] | write permissions for that path.
gitea-runner-1 | [Deploy Jekyll site/build] ❌ Failure - Main ---INSTALL GEMS ---
gitea-runner-1 | [Deploy Jekyll site/build] exitcode '23': failure

After a lenghty search I saw that Gemfile.lock wasn’t added to version control on my local machine because it is mentioned in .gitignore. So bundler in act_runner now tries to create it out of the given Gemfile without success as it only has read access to the checked-out dataset.

To solve this, I add Gemfile.lock to version control, removing it from .gitignore. As an additional benefit I make sure to have identical build environments locally and remote.

Dependency: `sass-embedded`

Less than 20sec into the build I get the following error:

Resolving dependencies...
Could not find gem 'sass-embedded (= 1.69.5)' with platform 'x86_64-linux' in
rubygems repository https://rubygems.org/ or installed locally.

After some more searching, I found a resolution: Specify a certain version of jekyll-sass-converter in the action or, alternatively, add the gem “github-pages” to the Gemfile. The latter manages to load a working version of CascadingStyleSheets (CSS) Preprocessor Sass.

Finally, bundle install finishes successfully:

✅ Bundle complete! 5 Gemfile dependencies, 43 gems now installed.

jekyll build

As I want act_runner to save the build output on my host machine, I add a Volume to Gitea’s docker-compose.yml:

# gitea/docker-compose.yml
# [...]
volumes:
 - ./runner/blog_staging:/opt/blog_staging
 - ./runner/data:/data
 - /var/run/docker.sock:/var/run/docker.sock

Let’s retry the build:

 Destination: /opt/blog_staging
 Generating... 
 Jekyll Feed: Generating feed for posts
jekyll 3.9.3 | Error: Permission denied @ dir_s_mkdir - /opt/blog_staging
/usr/local/lib/ruby/3.2.0/fileutils.rb:406:in `mkdir': Permission denied @ dir_s_mkdir - /opt/blog_staging (Errno::EACCES)
	from /usr/local/lib/ruby/3.2.0/fileutils.rb:406:in `fu_mkdir'

Also this problem can easily be solved: Either you build to /tmp, where the jekyll user has access. Or you hand over the output directory like so: chown -R jekyll /your/build/output.

With this change, I get successful action runs.

Transfer artifacts

Where do I find the build artifacts now? I don’t see any _site folder: Neither on the host machine’s volume, nor in the Gitea or Runner containers.

Docker volumes for `act_runner` cannot share build artifact

This seems logical as I learn the mechanics of act-runner: It will spawn an own action Dockerimage with own volumes that are associated by Task-ID.

DRIVER VOLUME NAME
--> local GITEA-ACTIONS-TASK-84_WORKFLOW-Deploy-Jekyll-site_JOB-build <--
--> local GITEA-ACTIONS-TASK-84_WORKFLOW-Deploy-Jekyll-site_JOB-build-env <--
local act-toolcache
local blog_staging

If I do not extract the build artifacts from the action run, they will be thrown away together with the action run when it has completed and everything is gone. The runner’s volume set up above doesn’t touch the action as it again executes in an isolated image.

To really understand this has cost me a lot of time and many failing action runs. Don’t repeat my mistakes and read the according section in Gitea’s documentation thoroughly.

Well, this turns out more complicated than I thought. As a single maintainer of my website, this is like taking a sledgehammer to crack a nut. Anyways, I’ll follow through.

But I can use `upload-artifact`, right?

Right? Well, at least I thought so. I modified my Action script accordingly and used the upload function from the Github Actions marketplace:

# workflows/jekyll-build-action.yml
# [...]
# Automatically upload the build folder to Giteas blog repo folder
 - name: --- UPLOAD ARTIFACT ---
 uses: actions/upload-artifact@v3
 with:
 path: /workspace/schallbert/blog/
 name: Blog_Staging
 retention-days: 2

Great, now I have a downloadable Zip file on Gitea’s web surface:

But how do I now get that downloaded to the server’s disk? The action is fully encapsuled and cannot even access its underlying Docker Daemon (for good reasons, I guess).

So I search for the assets in the Gitea-Container that persists the artifacts at/data/gitea/actions_artifacts/BUILD_ID. Unfortunately, they are a heap (hundreds) of .chunk.gz files with cryptic numbers as name where I don’t know how to merge into one single archive.

And there’s another thing I don’t like about upload-artifact: My website is becoming bigger and bigger due to an increasing amount of media that I use. That’s why the uploader requires nearly two minutes for compressing and packing - with a linear ascending outlook.

Dead end.

Also Docker `cp` won’t work here

Then I try using Docker’s copy operation to get the artifacts to my host machine:

# workflows/jekyll-build-action.yml
# [...]
- name: --- COPY ARTIFACT ---
 run: docker cp gitea-runner-1:/workspace/schallbert/blog /tmp/blog_staging

This fails like so:

gitea-runner-1 | docker cp gitea-runner-1:/workspace/schallbert/blog /tmp/blog_staging
[...]
gitea-runner-1 | [Deploy Jekyll site/build] | /var/run/act/workflow/3.sh: line 2: docker: not found
gitea-runner-1 | [Deploy Jekyll site/build] ❌ Failure - Main --- COPY ARTIFACT ---

This was expected as the action container knows no Docker. I cannot move files from “inside” the container to another one. Or can I? By forwarding the host machine’s Daemon docker.sock, according to Gitea documentation, I might be tempted… No, I will not continue here - see above, I’d puncture encapsulation if I did.

Transfer artifacts via SFTP

I’m running low on options. I think I could copy the artifacts via SFTP from within the act_runner’s action image into the Webserver-Container¹.

Luckily, there already is an Action for this: scp-files. Now I have to modify the webserver’s container in a way to be accessible via SCP, inhale an SSH-key, and save incoming artifacts.

On the other hand, people in multiple forums say that this is not a clean solution. “Ac container should only accommodate one application. But there’s a solution again: A docker container with only the SSH-Daemon installed. If I have a shared volume between that one and my server, I might be successful.

Looks like more work still. Another container, piping files out and back into my system - that just doesn’t seem right. So I continue my research.

Handover a volume to the Action-Container

All these failures and dead ends have been with me for over a month now. I still really want to be able to build, test and publish automatically. I’ve learned a lot along the way and now I’m hoping that this approach will finally get me to the solution.

During my search I came across the option to change the configuration of the runner. That way, I might end up including a volume that I can share between the action and my server via the daemon on the host.

If that works, I wouldn’t have an additional security risk like a publicly available file transfer container or the long waiting times caused by upload-artifact. So let’s get to work! There are more reasons why volumes can make sense in the respective action - so there must already be people out there who have managed this.

So I add a volume to the Action script:

# workflows/jekyll-build-action.yml
# [...]
container: 
 image: jvconseil/jekyll-docker:latest
 volumes: 
 - /tmp/blog_staging:/blog_staging
 - /opt/cache:/opt/hostedtoolcache

But the artifacts still don’t show up on my host’s filesystem. I strip down the action as much as I can to just check if the volume creates a folder in the action container:

# workflows/jekyll-build-action.yml
# [...]
steps:
 - name: --- CHECK_VOLUME ---
 run: |
 ls -al /blog_staging

But still:

gitea-runner-1 | [Deploy Jekyll site/build] [DEBUG] Working directory '/workspace/schallbert/blog'
gitea-runner-1 | [Deploy Jekyll site/build] | ls: /blog_staging: No such file or directory
gitea-runner-1 | [Deploy Jekyll site/build] ❌ Failure - Main --- CHECK_VOLUME ---

Some more, fruitless tries (maybe I had misspelled anything?) and extensive research in multiple forums I found out that there is a valid_volumes attribute in act. If I don’t have the volume added here, it won’t bind to the container.

OK, let’s add it to the action script:

# jekyll-build-action.yml
# [...]
container: 
 image: jvconseil/jekyll-docker:latest
 valid_volumes: 
 - '**' # This does not work. Also specialized lists indicating the volumes directly won't work
 volumes: 
 - /tmp/blog_staging:/blog_staging
 - /opt/cache:/opt/hostedtoolcache

Still nothing. At least, now, I get a warning in the logs:

gitea-runner-1 | [Deploy Jekyll site/build] [/tmp/blog_staging] is not a valid volume, will be ignored
gitea-runner-1 | [Deploy Jekyll site/build] [/opt/cache] is not a valid volume, will be ignored

Something is wrong with the transfer of this option. There are reports in which the registration of the volumes works. So I take the instructions and create a configuration file where I enter the following:

# runner/config.yml

# Volumes (including bind mounts) can be mounted to containers. Glob syntax is supported, see https://github.com/gobwas/glob
 # You can specify multiple volumes. If the sequence is empty, no volumes can be mounted.
 # For example, if you only allow containers to mount the `data` volume and all the json files in `/src`, you should change the config to:
 # valid_volumes:
 # - data
 # - /src/*.json
 # If you want to allow any volume, please use the following configuration:
 # valid_volumes:
 # - '**'
 valid_volumes: ["/tmp/blog_staging", "/opt/hostedtoolcache"]

Important: The notation must be a string, comma-separated, and always has to specify the Source of the volume. In other words, the part that comes before the :. However, the error message “not a valid volume” still appears when building.

But then I realize that the config must be made available to the act_runner itself as a volume - otherwise the runner running in the container cannot access it at all!

So my docker-compose.yml for Gitea, section “runner” now looks like this:

# gitea/docker-compose.yml
# [...]
 runner:
 image: gitea/act_runner:latest
 environment:
 - CONFIG_FILE=/config.yml
 - GITEA_INSTANCE_URL=https://git.schallbert.de
 - GITEA_RUNNER_NAME=ichlaufe
 - GITEA_RUNNER_REGISTRATION_TOKEN= <redacted>
 volumes:
 - ./runner/config.yml:/config.yml
 - ./runner/data:/data
 - /opt/hostedtoolcache:/opt/hostedtoolcache
 - /var/run/docker.sock:/var/run/docker.sock

And, finally:

gitea-runner-1 | ls -al /blog_staging
gitea-runner-1 | [Deploy Jekyll site/build] | total 8
gitea-runner-1 | [Deploy Jekyll site/build] | drwxr-xr-x 2 root root 4096 Dec 27 07:32 .
gitea-runner-1 | [Deploy Jekyll site/build] | drwxr-xr-x 1 root root 4096 Dec 27 07:46 ..
gitea-runner-1 | [Deploy Jekyll site/build] ✅ Success - Main --- CHECK_VOLUME ---

What an act. I’m so glad that everything is running smoothly now and that I can actually find the build files on my local host system! 🥳

One last tip

If you work with a reverse proxy like I do and get strange connection refused error messages when starting up the runner like:

docker compose up
[+] Running 2/0
 ✔ Container gitea Created 0.0s 
 ✔ Container gitea-runner-1 Created 0.0s 
Attaching to gitea, gitea-runner-1
gitea-runner-1 | level=info msg="Starting runner daemon"
gitea-runner-1 | level=error msg="fail to invoke Declare" error="unavailable: dial tcp <address>: connect: connection refused"
gitea-runner-1 | Error: unavailable: dial tcp <address> connect: connection refused
gitea | Server listening on :: port 22.
gitea-runner-1 exited with code 1

Then your reverse proxy is either not correctly configured or - like in my case - shut down.

Using SFTP (Secure File Transfer Protocol) which is file sharing per ssh (secure shell), I’d increase attack surface of my system just a bit. This is because the SSH client would be reachable for anyone and not just for my action. ↩︎

🗙 Gitea Actions - Jekyll Workflow

Schallbert — Thu, 30 Nov 2023

After setting up my own little server in this project and adding all necessary applications to serve my blog, I now want to automatically build, integrate, and deploy.

Desired outcome

This is how I want it to be: When my Gitea instance detects a git push to the remote origin of my blog’s repository, I want an action runner to start a Docker container. That container shall load Ruby, Bundler, Jekyll, and all other required dependencies. Then it shall checkout the blog’s source files and then run the bundle exec jekyll build command. When its output is ready in the _site folder, I want the runner to self-terminate and to free all occupied resources.

At this point I’d like to manually check my blog’s integrity, maybe also add some automated checks like detecting broken links or similar.

Finally, I want to be able to have the site deployed and thus made available to the public with another simple command.

Starting point: Github Actions

As usual, I first search other people’s solutions for the same problem. I found the Github Actions starter-workflows Repository, created by Github themselves, and copied most of it into my action as a first shot:

# jekyll-build-pages.yml
# Sample workflow for building and deploying a Jekyll site
name: Deploy Jekyll site
run-name: ${{ gitea.actor }} builds Jekyll site
on: [push]

jobs:
 # Build job
 build:
 runs-on: ubuntu-latest
 steps:
 - name: Checkout
 uses: actions/checkout@v3
 - name: Setup Ruby
 uses: ruby/setup-ruby@55283cc23133118229fd3f97f9336ee23a179fcf # v1.146.0
 with:
 ruby-version: '3.1' # Not needed with a .ruby-version file
 bundler-cache: true # runs 'bundle install' and caches installed gems automatically
 cache-version: 0 # Increment this number if you need to re-download cached gems
#[...]

After uploading this workflow to my blog at workflows/jekyll.yml, the runner indeed started its work:

Problems installing Ruby

That was a quick win. On the other hand, the runner wasn’t even able to successfully install Ruby. Here’s the corresponding log:

::error::The current runner (debian-11-x64) was detected as self-hosted because the platform does not match a GitHub-hosted runner image 
(or that image is deprecated and no longer supported).
In such a case, you should install Ruby in the $RUNNER_TOOL_CACHE yourself, for example using https://github.com/rbenv/ruby-build
You can take inspiration from this workflow for more details: 
https://github.com/ruby/ruby-builder/blob/master/.github/workflows/build.yml

The error message looks reasonable and is well-placed.

Background: My runner operates in Docker which uses a stripped-down Debian image named node:16-bullseye (Referenz). Looks like it is incompatible with the Github Actions Runner which executes atop Ubuntu22.

There are quite some options to resolve:

Troubleshooting options

Run the runner on Ubuntu22, use labels for that.
Don’t use Docker but have the runner operate on the host’s operating system
Use prebuilt Ruby, installed in the runner’s cache
Use an out-of-the-box Jekyll-Dockerimage to not require additional installation steps

Operate the runner with Ubuntu22 instead of Node16

There is a matching Dockerimage, but it will take a lot more disk space and RAM than a minimalistic one. Plus, it takes a little more time until it is live. Let’s see whether there is a more simple solution.

Run the runner on the host’s operating system

This has considerable security drawbacks as unprotected branches might allow third-party injected runners via git push --force. In an extreme case, it would blindly execute malware right on my server system. In addition, I’d kill portability by hardwiring the runner to a machine’s OS.

Use pre-built Ruby in runner toolcache

As proposed by the error message, I followed another manual to map a volume containing prebuilt ruby on my host using Gitea’s docker-compose.yml. In addition, I assigned the targeted folder to the RUNNER_TOOL_CACHE environment variable:

# gitea/docker-compose.yml
# [...]
runner:
 image: gitea/act_runner:nightly
 environment:
 - GITEA_INSTANCE_URL=<redacted>
 - GITEA_RUNNER_NAME=<redacted>
 - RUNNER_TOOL_CACHE=/opt/hostedtoolcache
 - GITEA_RUNNER_REGISTRATION_TOKEN= <redacted>
 volumes:
 - ./runner/data:/data
 - /opt/hostedtoolcache:/opt/hostedtoolcache
 - /var/run/docker.sock:/var/run/docker.sock

Here, I map the host’s filesystem path /opt/hostedtoolcache to Docker’s /opt/hostedtoolcache. Docker creates the folders automatically “on both sides” if they are not yet existing on next docker compose up.

Let’s try to install Ruby on the host machine.

So I download the ruby-build repository via git clone and have the installer run ./ruby-build/install.sh. Then, I execute the build with the requested target path from the error message:ruby-build 3.1.4 /opt/hostedtoolcache/Ruby/3.1.4/x64

==> Downloading openssl-3.1.4.tar.gz...
 % Total % Received % Xferd Average Speed Time Time Time Current
 Dload Upload Total Spent Left Speed
100 14.8M 100 14.8M 0 0 81.3M 0 --:--:-- --:--:-- --:--:-- 81.1M
==> Installing openssl-3.1.4...

BUILD FAILED (Ubuntu 22.04 on x86_64 using ruby-build 20231114)

Problems with prebuilt-Ruby

The logs shows the problem:

[...] No C compiler found, please specify one with the environment variable CC [...]

So I type apt install build-essential and confirm with which gcc that I now have a C compiler installed at /usr/bin/gcc Let’s try again!

crypto/comp/c_zlib.c:36:11: fatal error: zlib.h: No such file or directory

Hmm, let’s load it via apt install libz-dev and re-run the insall. This time it takes a while, then:

*** Following extensions are not compiled:
openssl:
 Could not be configured. It will not be installed.
 /tmp/ruby-build.20231205114312.9174.0gmhNf/ruby-3.1.4/ext/openssl/extconf.rb:100: OpenSSL library could not be found. You might>
 Check ext/openssl/mkmf.log for more details.
readline:
 Could not be configured. It will not be installed.
 /tmp/ruby-build.20231205114312.9174.0gmhNf/ruby-3.1.4/ext/readline/extconf.rb:62: Neither readline nor libedit was found
 Check ext/readline/mkmf.log for more details.
*** Fix the problems, then remove these directories and try again if you want.
[...]

OK, openssl this time. Get it with apt install libssl-dev and apt install libreadline-dev and finally :

Installed ruby-3.1.4 to /opt/hostedtoolcache/Ruby/3.1.4/x64

!🎉!

Still no success with the action

Strange, it is the same error all over again:

💬 ::debug::isExplicit: 3.1.4

gitea-runner-1 | [Deploy Jekyll site/build] 💬 ::debug::checking cache: /opt/hostedtoolcache/Ruby/3.1.4/x64
gitea-runner-1 | [Deploy Jekyll site/build] | ::debug::checking cache: /opt/hostedtoolcache/Ruby/3.1.4/x64
gitea-runner-1 | [Deploy Jekyll site/build] 💬 ::debug::not found
gitea-runner-1 | [Deploy Jekyll site/build] | ::debug::not found
gitea-runner-1 | [Deploy Jekyll site/build] ❗ ::error::The current runner (debian-11-x64) was detected as self-hosted

This results from the following Action code:

// setup-ruby/ruby-builder.js
if (common.shouldUseToolCache(engine, version)) {
 inToolCache = common.toolCacheFind(engine, version)
 if (inToolCache) {
 rubyPrefix = inToolCache
 } else {
 const toolCacheRubyPrefix = common.getToolCacheRubyPrefix(platform, engine, version)
 if (common.isSelfHostedRunner()) {
 const rubyBuildDefinition = engine === 'ruby' ? version : `${engine}-${version}`
 core.error( [...] )

Check docker volumes

Hmpf. Let’s see whether Ruby is really available in that volume. To do this, I use docker ps to get the runner’s container ID and enter it here:

docker exec -it <containerID> bash

Then I navigate to cd /opt/hostedtoolcache, list its contents with ls and there it is: Ruby. It is there but still cannot be found in the Action.

Accept my failure

No way to proceed for me at this point 😖. I opened this problem to the community as act_runner cannot find hostedtoolcache and will try

Another option.

Root cause

Looks like the support of setup-ruby action for self-hosted runners like mine might have been discontinued. Maybe the documentation, creating the above error message, has not been updated accordingly.

Multi-language site - l10n

Schallbert — Tue, 14 Nov 2023

Providing a website in different languages is called “internationalization” - in short i16n or “localization” - l10n. If you search these abbreviations in the web, you’ll find a lot of useful material. By the way, the numbers in the brief version stand for the letter count in the word’s center.

Motivation

Right from the start, I was asked to prepare a German version of this website. To date I considered this too much effort and this only changed half a year ago when I was verbally asked by multiple people independently from each other.

So I invested half a year of evenings to get the heavy lifting done, transcribing roughly 80 articles. But I still wasn’t quite done.

Jekyll extension

I looked at a couple of extensions for Jekyll dedicated to multilingual support like demondehellis’s plugin-free solution, untra’s polyglot plugin, kurtsson’s jekyll-multiple-languages-plugin and some other solutions to publish a website in multiple languages.

So I had to decide for one. The plugin-less solution was out quickly because I wanted a concise naming convention where identical posts in different languages have identical names. Plus, I wanted interlinks to be persistent for the selected language. Something would have to keep language pots separated.

After some research, I found leo3418’s wonderful series of blog posts for implementing Polyglot - and so I decided to go with that one.

Polyglot: Installation

Installation of PolyGlot is simple because both the documentation in the repository’sreadme.md and leo3418’s related post do instruct these first steps in detail. They do this so well so I won’t lose another word about it here.

Restructuring the website

Polyglot offers two possibilities to adapt the site’s structure for multilanguage support:

Use the language abbreviation in the post’s filename like this: 2023-11-14-jekyll-polyglot-language-support-en.md, repeat for each article in a different language with the appropriate abbreviation
Create folders named like the language code, e.g. /en and put all english content in there, but keep file names the same between languages.

I decided for the latter option out of laziness - I didn’t want to rename existing articles. Polyglot manages to stick to the same language when navigating a site. If you, for instance, click https://schallbert.de/milling-small-parts/, Polyglot will assume you want the English version to be presented and thus provide it. Where this doesn’t work anymore is section titles as they are translated themselves, invalidating attached links, so the section link #restructuring-the-website will lead to a 404 fault when called in the German version.

Thus, when using section titles, the full post’s path has to be noted.

This is getting a little more complicated. Jekyll’s _config.yml is intended for one version of the website only, so we cannot just create two variants of this file and expect it to work. To still enable full translation, we’ll have to overwrite language sensitive parameters like site.title or site.description externally depending on selected locale.

I have solved this adding l10n.yml files in the _data folder that covers these parameters for each language, and then packing them into a language-code folder like we’re used to do with our posts. To show their content, the relevant liquid parameters in html files (_includes folder) have to be modified to now search there instead of taking the standard config. Example: {{ site.data.ui-text[site.data.l10n.locale].follow_label | remove: ":" | default: "Follow" }}. German version of this site will show FOLGEN: while the English one says FOLLOW:.

Language selector

Of course I want to offer the option to select a locale on every page. To do this, I added the language code and its flag to the footer. On click, it will reload the page in the requested locale. The following code I wrote in Liquid enables this behavior:

{% raw %}
{% for lang in site.languages %}
 {% assign url_parts = page.url | split: "/" %}
 {% assign lang_code = lang %}

 {% assign collection_name = url_parts[1] %}
 {% assign post_url = url_parts | last %}

 {% if lang == site.default_lang %}
 {% if collection_name == post_url %}
 {% assign lang_url = "/" | append: post_url %}
 {% else %}
 {% assign lang_url = "/" | append: collection_name | append: "/" | append: post_url %}
 {% endif %}
 {% else if url_parts.size > 2 %}
 {% assign lang_url = "/" | append: lang_code | append: "/" %}
 {% if collection_name == post_url %}
 {% assign lang_url = lang_url | append: post_url %} 
 {% else %}
 {% assign lang_url = lang_url | append: collection_name | append: lang_url | append: post_url %}
 {% endif %}
 {% endif %}
 <a href=" {{ lang_url }}">{{ site.data[lang].l10n.lang_name }}</a>
{% endfor %}
{% endraw %}

Here I’m splitting the current page url. The last section will be the post_url. The algo assumes that the first part is the post’s collection name, e.g. posts-hardware. The following logic inserts the language code into the url if and where needed so that the correct full url is built depending on the used collection and page navigation depth.

Publishing

I was pretty proud of this elegant way of selecting languages. I thing it all fits in nicely and the multi-language support potentially adds a good number of readers. The big downer is the duplicated work for creating each post in two languages.

Unfortunately, the site now wasn’t compatible anymore with Github Pages - the auto release would show each page twice instead of keeping them separated by language. Two options remained: creating custom Github Actions to publish on Github Pages, or to move the whole site to an own server. I’ll decide for the latter to be on the safe side regarding data protection regulations because in that case I can make sure the hoster is situated in Germany.

Well, this upcoming project is too big to take on in this post.

Milling small parts

Schallbert — Wed, 01 Nov 2023

Why milling?

It is sometimes hard to manufacture small parts in low quantities, especially when their dimensional tolerances have to be low. In this case, die casting wouldn’t be economical. Hobbyist 3D-printing machines wouldn’t deliver the low tolerances I need and affordable lasers wouldn’t be able to cut parts that are 2.5D (i.e. that don’t have an even surface). Plus, I’m not a fan of ordering parts from sweat shops half a globe away.

But still, milling these parts is perfectly possible. This video shows how:

Milling Small Parts↗

What accessories are required?

All you need is:

A vacuum table
Vacuum pump, preferably supporting deep vacuum levels
Some plastic wrap
A Vacuum fleece

What is that part for?

The part I’m fabricating in the video belongs to one of my soap butlers. The current version can be mounted to any shower rail with diameter 18-25mm, and needs this small part for clamping securely to the rail. It is in contact with the rail from the rear and both locks the soap butler in place, limiting its possible “wobble” movement in Z-direction.

This way, not only soap bar shampoos can be placed here, but also much heavier shampoo bottles.

The part has to fit in snugly as it shall be able to slide without noticeable play, but still has to move effortlessly to adjust to different shower rail diameters.

What else do I need to consider?

plastic wrap: make sure the cutouts you do are not too big. Else, the plastic wrap will be pulled in by the endmill, wrap it, and leave marks on the workpiece. It imbalances the cutter and may cause additional damage this way. To avoid this, use double-sided tape to secure the wrap where you do the cutouts.
vacuum fleece: Note that the fleece will be compressed when you apply vacuum. Z-zero height of workpiece surface has to be taken with vacuum applied to get accurate results.
Don’t cut all the way through. Leave an onion skin. Its strength depends on the material you work with. For PMMA I use 0.3mm, for wood 0.7mm.
I don’t account for wrap strength. I know that thicker protective wraps e.g. for PMMA and Dibond have a strength of 20-30µm (where I calculate with it), so I’d assume the plastic wrap I use is less than 10µm which is in the magnitude of machine accuracy.

DIY edge finder

Schallbert — Sun, 21 May 2023

As a CNC operator, I sometimes do have to know where my workpiece ends. Therefore I need its edges’ X/Y coordinates. I was kind of solving this problem to date by using a tapered tool which I would manually position above the workpiece corner the best I could, visually controlling its position with help of my phone’s camera zoom.

The problem

This is relatively quick to setup and accuracy is high enough for most jobs. But it depends on the line of sight where even slight deviations create large errors so you have to be pretty thorough. When I started doing double-sided workpieces like this, I felt the accuracy just is not high enough anymore.

Solution

I was discussing my problem with the machine manufacturer, thinking they might hint me towards buying an edge detector “3D finder” or similar product. But wrong I was! Instead, I was able to build an edge finder that did not cost me anything at all and in no time.

Method

Roy was like “Well, really the only thing you need is a ball bearing. Put it onto a pole and mount it like any other tool. Have the spindle turn at low RPM. Then slowly have the machine move towards the workpiece edge. Once the outer ring of the bearing stops, you’ll hear and see it, and stop the machine movement. That’s the position you are searching for. Offset it with the bearing’s radius and you’re done.”

The build

I was intrigued by that idea. As I don’t have access to a good hardware store that sells bearings and stuff nearby, I was searching the household. And voilà, my kids own a fidget spinner they didn’t use for a while. So I… snitched it. It contains a 22mm bearing with 8mm bore. Ok next - what should I use as a shaft?

I went for a dull drill bit, cut it in half, shoved the bearing on and added a drop of glue to secure it in place - Done!

Execution

Mount the edgefinder and jog the machine close to the workpiece you’d like to measure.
Have the spindle turn at low RPM, e.g. S500
Optional: To keep things simple, switch to machine coordinates
Make the machine move towards the workpiece, e.g. with command G01 X5.0 F1
Prepare to hit “routine stop” once the bearing stops turning
Note down machine coordinates of where you stopped or set G92 offset to here
Repeat with other axes if required

Your browser does not support the video tag.

Simple edgefinder in action.

Hardware Test

Schallbert — Sun, 23 Apr 2023

A new drive

Recently, I bought a spindle upgrade. I now switched to a 3-phase induction motor with forced air cooling that requires a Variable Frequency Drive so I can set different speeds. The system takes the same analogue 0-10V signal as my previous motor. For the numerical controller it looks the same with one exception: The VFD can now create an emergency stop signal if something goes wrong.

As you can imagine, the new spindle subsystem is by far more complex than the old one. I had to buy, wire, and setup a dedicated electric control box.

So why testing?

With higher complexity comes higher risk of failure. There are more electrical and mechanical parts involved that each can behave incorrectly or even be destroyed if connected improperly. As I am just a human being who makes mistakes, I’d better not jump in at the deep end. So, while wiring everything up, I wondered how such a system should be systematically tested.

Test Strategy

I borrowed the different test scopes from my job in the software industry. I think they fit in quite well:

Unit: Test that verifies behavior of a component of the system in isolation. All external influences are removed or mocked away. Example: Check if the blower turns if supplied with its nominal voltage.
Integration: These tests integrate different components that provide a common functionality. Example: Verify that spindle cooling works with delayed shutdown.
Subsystem: Test that verifies the correct interaction of all components within a subsystem. Example: The spindle speed changes when feeding the analogue input of the VFD with different voltages.
System (End2End): Tests that verify a system from the end-user perspective. Example: When pressing the emergency off switch of the CNC, does the VFD bring the spindle to a halt safely?

Verify early

From my experience, it makes sense to perform tests at the earliest possible point in time. After the spindle was delivered, I was able to check right away whether all bearings were smooth-running, if the fan would turn when voltage was applied, and see whether the motor windings were within expected resistance range, and so on. I could do these unit-level tests even before I started building or wiring the control box.

Your browser does not support the video tag.

Unit test: Spindle cooling fan

This might save a lot of troubleshooting when something doesn’t seem to work correctly during comissioning. And if you find something this early, you can directly get in touch with the manufacturer of the affected component to request an exchange while not being blocked on the other end.

Bottom-up testing

The four levels of testing are my verification strategy here. I try to have many simple and quick early test on unit level, and just a few complex system tests that each involve a multitude of preparation and test steps, being executed just once upon initial startup when the build is complete.

When you look at the concept of the test pyramid that is used a lot in software testing, it may be applicable for hardware tests as well to some extent, maybe with a different number of layers and varying names for the levels of integration.

Test plan

I don’t usually write test plans for hobby projects because the systems I design are rarely as complex as this. Plus, most of them do not require dangerous voltages to operate unlike this one.

To better visualize the system, I drew a layout of all components with the most important inputs and outputs.

From that image, I derived a test plan. I tried to cover each component’s interactions with each other, ending up in more unit than system test cases.

Your browser does not support the video tag.

Unit test: Supply +24V rail OK

Here I checked whether the cooling fan power supply operates correctly.

Your browser does not support the video tag.

Unit test: Time delay setting OK

With this test I verify that the cooling fan relay configuration and activation work as expected. For the purpose of this test, I selected shorter delay times than in the later application.

Your browser does not support the video tag.

Integration test: VFD initial operation

The variable frequency drive is able to turn the spindle in the correct direction of rotation at different speeds.

Test execution

Of course, there were many more hardware-based tests that I didn’t mention in the above plan. Things like is the spindle correctly aligned to the Z-axis? Have I fastened all clamps with the appopriate torque? Are all connectors inserted and locked?

Before trying to run through material on the CNC I performed all the tests and actually found one issue on the system level: I had a pinning error in the spindle speed analogue output of the numerical controller’s signal harness so that the spindle motor system wouldn’t start despite the run signal being present.

This just took minutes to figure out because I had subsystem-tested the VFD before (Test case: “Reacts to speed command”), knowing that it cannot be the culprit.

Documents

In case you plan on adding a VFD-controlled spindle motor to your machine, please find my full test plan below.

VFD-controlled spindle motor test plan

But please take this plan with a grain of salt and note the usual disclaimer:

⚠️ Risk of electric shock. ⚠️

This kind of system should only be built or worked on by professionals. Especially avoid touching live parts or VFD components before the VFD’s bleeder resistors have had enough time to discharge the intermediate circuit’s bulk capacitors.

Dibond Tests

Schallbert — Fri, 10 Mar 2023

Note: The engravings I disuss in this article are relatively fine. Font size is between 3mm and 6mm, and workpiece size is 80x80mm. Viewed at a distance of an ell or more, the quality issues are not apparent and the overall impression is quite good. Still, I wanted to improve.

Further quality tuning on Dibond

After I got mediocre results with my previous milling project, I decided to do some more test runs in Dibond. Before I started those, I noted down possible causes for rough cutting edges that I saw earlier.

Then, I’ll try to rule out noise factors to track down root causes, and in a further step I will do my best to eliminate most of the issues.

Possible causes for poor quality

Worn-out or dull cutter
Cutter geometry
Improper Feed / Speed parameters
Climb milling vs. conventional milling
Too shallow engrave depth
Machine vibrations on portal acceleration and/or deceleration, especially on Y-axis
Router motor mount not stiff enough
Imbalanced cutter / collet

Cutter inspection

I’m not an expert on cutters, but I cannot imagine that a carbide bit is worn out after half an hour of cutting through 0.3mmthick AlMg1 which is the harder compound of Dibond composite material. Also, the cutting edge seems to be in a good shape. The only thing I noticed was that there seems to be a thin layer of Aluminium sticking to the tip of the bit. Maybe this affects cut quality?

Carefully, I removed the aluminium residues with a trimming knife and minimal sanding. As good as new!

Cutter geometry

The cutters I’m using for this test do not have chip flutes to guide shavings away from the place of cut. There are others on the market which do provide one and I wonder if it makes a difference. Maybe a spiral flute could improve quality of cut?

Background behind that thesis is that for cutting letters, my CAM won’t use the defined dive ramp but plunges straight into the material. When you have a closer look at the images below, you might even see marks in the clearcoat around the place where the cutter hit Z0, shearing the surface away but not yet expediting the chip away from the cut.

On the other side, a spiral flute will try to pull the aluminium layer upwards. As it is “only” thermally bonded to the polyethylene core, adhesive force might be too low, especially with fine engravings or letters like “e, a, o, R, B…”

Summary: Buy spiral flute engrave cutters, try them, and document the results 😅

Improper Speed / Feed parameters

I double-checked required spindle RPM for my engrave cutter which is single-flute. I again got a value higher than my maximum speed of 24000RPM, so I can confirm to use my motor’s maximum speed with that bit.

To try out other feed rates, I created a quick 2D design in my CAD and had my CAM run it in different feeds: F600mm/min, F800mm/min, and F1000mm/min.

There are a lot of things that we can observe here:

Look at the labels for the different feed rates. The font is so small, that the engrave cutter won’t even fully cut through the upper Aluminium layer. It rather “mashes” the aluminium into the polyethylene layer below, and squeezes the aluminium sideways.
Although I change the feed rates by as much as 20% per spiral turn, quality of cut remains the same. This confirms that for my 6mm 0.5tip 90° cutter in Dibond, a feed rate of F1000mm/min is fine.
It is clear to see that the “outer” edges of the engrave operation are rough. I can feel this when I stroke the workpiece with my finger.
The “inner” edges of the engravings are really smooth, though. They gleam in contrary to their matte counterparts.
The black, inner parts of the spiral are shaped very even and without cutter marks or a lot of remaining chips.
The capital “T” of the word test runs looks a bit strange as there are vertical marks. Let’s come to this later.

Summary: Speed and feed parameters are fine and don’t seem to be the culprit for poor machining quality in this case.

Climb milling vs. conventional milling

To get a clear view on the effects of milling strategy, let’s take a detailed look at the spiral.

It is easy to see that the outer cut has very clean edges while the inner one has not. The spiral has been milled outwards to inwards, and the spindle turns clockwise. The groove was cut in a single pass for the narrower line, and with a smoothing increment of Z-0.1mm for the wider one.

This means that conventional milling has been applied to the outer edge of the groove, and the inner edge was machined with climb milling, i.e. the cutter was moving in the direction of cut and not against it.

Summary: Although climb milling is preferred on a CNC for various reasons, this is not true for aluminium composite materials like Dibond. Here, conventional milling is the strategy to choose if you want clean chamfers.

Too shallow engrave depth

Summary: As the images above show, it is a problem if the aluminium layer is not fully broken through. In some follow-up experiments I tested a minimum viable engrave depth and ended up with a value of ~150% of the aluminium layer’s strength, in my case that is 0.45mm.

Note that this constraint might only be valid for V-cutters that don’t feature spiral cuts or cutter flutes.

Too high engrave depth

This was a guess that came up when looking at the images from my previous experiment with Dibond. To compare, I created two spiral engrave paths with different depths of Z-0.5mm and Z-1.0mm. While the former wouldn’t make any difference when I add another pass for smoothing, it makes the latter engrave look a little more clean.

Summary: For deeper engraves, smoothing runs are helpful but not a must. Instead, a much bigger lever to improve quality is to use conventional milling only.

Machine vibrations

Remember the test texts carrying strange marks? At the beginning I thought this was due to the brushed-look surface: It is not actually brushed but imprinted under high pressure using rollers. But when I had a closer look, another assumption came to my mind: Machine vibrations.

The texts have been made with different feeds, where 6 indicates 600mm/min and so on. Below on the right I have prepared a second image that has an extra zoom into the “t"s for better comparison of the feed rates. Actually, the quicker engrave settings look better than the slower ones.

I have been excercising the limits of my machine a lot in the past half a year. I came up with pretty high values on acceleration and maximum velocity which I verified the machine was capable to work with. “Unusually high” is actually what the manufacturer of my controller software said when I contacted their support with an unrelated issue about a warning message “Velocity was higher than max!” a couple of weeks ago.

That said, to verify my hypothesis I created yet another test sample in CAD and CAM where I’d only carve Test texts and straight lines with sharp corners so that the machine has to accelerate and decelerate to full carve speed and back to zero more often.

This time I’d keep the feedrate constant at f1000mm/min but change the machine’s acceleration values to see whether the acceleration unsettles the machine’s portal system in a way that shakes are propagated to the tool tip.

Testergebnisse: Detailaufnahmen von Stichel und Material bei variabler Maschinenbeschleunigung.

The 6mm 90° 0.5m tip V-cut tool. At the cutting edge, there's a little built-up material.

It has been used for roughly 30minutes. The cutting edge looks intact.

Minimal residues of Aluminium seem to stick to its tip.

Also here, the tip colour is different than the rest of the bit.

Result with high machine acceleration

Result with medium machine acceleration

Result with low machine acceleration.

You can see that the vertical marks become less visible on lower acceleration settings. As I had to reset the machine after each parameter change, I must re-measure Z0 height everytime. On the lowest acceleration setting, I probably had a grain of dust below the sensor as the depth of cut is visibly less than on the other two test samples which deteriorates the engrave quality.

Question: “Did the machine reach target speed at all?” Text height is s0=6mm, so for example at a=800mm/s², the machine takes t=f/a=20ms to accelerate to its target feed rate. Travelling a distance of s=a/2*t^2=0.17mm while accelerating, the machine will run s1=6-2*0.17=5.66mm with target speed within the vertical stem of letter t. This proves that the machine actually gets to run its targeted feedrate. Basic kinematic laws come in handy when working with CNC machines from time to time.

Summary: Make sure your machine runs in a fine-tuned parameter envelope relative to its size, stiffness, weight, and control variables so you avoid judder.

Router motor mount

I know this could be a problem from my past experiments where at some point I overloaded my machine. But for engraving, the spindle runs at just 20% of maximum load.

Summary: This is not an issue here.

Imbalanced cutter / collet

Summary: If this was the case, I would have either expected strange sounds or vibrations emanating from my machine. Maybe also edges with irregular shapes or torn chips could be a side effect. Neither of that is the case, so I’d exclude this noise factor here.

Process and machine updates

Here’s what I actually did:

Inspect and clean cutters after usage
Try to avoid climb milling in dibond. Rather have the engrave path a little wider, and have both edges of the engrave cut conventionally.
Add a smoothing run when engrave depth exceeds Z-0.6mm.
Reduce machine’s maximum acceleration values to minimize cutter marks.
Prepare a tryout CAD file to experiment with for optimal feeds and speeds when trying new materials.

See for yourself how the quality improved after applying all these changes.

Working with Aluminium composites

Schallbert — Thu, 02 Mar 2023

Experiments with Dibond

A friend of mine is currently starting a business. He wanted something to give away to his customers, something that they would remember so they’d return to him. It should not be the standard pen or cup, but something that is somehow more close to what he is doing. Something metally. Something Like a business card, but with a three-dimensional feel. Something engraved.

While searching for material on the internet, I came across aluminium composites - a sandwich material made of two thin sheets of aluminium with a core of polyethylene, also known under its trade names Dibond®, Dilite®, Alcubond® etc. It is available coated, blank or even coloured and, optionally, in different finishes.

“Its PE core is black, so it might be an ideal material for engraving”, I thought.

So I ordered some samples and showed them to my friend. He was blown away, and that’s where this story starts.

The Coaster Concept

We brainstormed about what to do with that material. It should be simple to design and not too large so it could be an easy giveaway.

I know that aluminium composites can be machined in a way that they form 3-dimensional objects (youtube video link). But this seemed to complicated for a start. Our next idea was the one that I’m discussing here:

Engraving Coasters. Round or rectangles with smooth corners, even custom shapes are possible.

I started some tries with engraving cutters that looked promising - but not perfect. The aluminium layer tends to smudge and wants to evade the cutter so it presses into the soft polyethylene core when diving into the material at a 90° angle. Still, my friend wanted me to proceed so I bought a couple of sheets and continued experimenting.

Cutters, speeds and feeds

As the coasters are below 100x100mm, I am using small endmills:

		[mm]	[RPM]	[mm/min]	[mm]	[mm]
acryl 30° upcut	1	2	24000	1600	800	3
engrave 90°, 0.5mm tip	1	6	24000	1000	500	1.5

The values may seem a bit aggressive, but I think I need to have a certain speed to not melt down the PE core. Chips do look good - short and thick - with these values and the machine gives a confidential hum when working, so I assume all is fine.

The first part

I cut the coaster holder first. It consists of four parts that can simply be put together. They don’t need glue as clearances are tight. I use a fence as orientation for the XY-zero of the workpiece as I want to do two-sided milling. This enables me to do the cutouts as a last step, keeping vacuum pressure on my table until that very last step. Plus, the idea was that I wouldn’t have to manually rework the parts.

The first try was rubbish - front and rear milling paths weren’t perfectly aligned, and as little as some tenths of a millimeter make a great difference here, so I had to fix the root cause (edges on the fence I did not notice before that I had to hone down) and try again.

Here’s the result:

I’m pretty happy how this turned out. I didn’t have to do any manual rework, it just fell out of the CNC like this. But I noticed that the chamfers I added to give the material a smooth touch were pretty different. While the ones on the “rear” that only cut 0.3mm off the edge where the upcut had previously done the part’s cutout were really good while the ones on the other side, where the tool had to cut 0.85mm deep into the material to add the chamfer, were a lot rougher (but still acceptable).

A batch of six

Then I selected a “simple-to-manufacture” logo - the Bitcoin Lightning symbol - and planned some basic engrave paths to have it carved. Here, I also did two-sided milling, which would not fully cut through the material but would leave an “onion skin” at the bottom which would be removed at a second pass on the bottom side while chamfering the edges.

I needed roughly 20 minutes to complete the jobs. Unfortunately, the fence I use to have a defined part edge that would stay in position even if turned around to mill the second side proved to be low repeatibility in terms of accuracy. The cutout on the bottom was off just one or two tenths of a millimeter, which in my case would double the total misalignment when turning it over its diagonal axis. That way, a part of my coaster’s outline had a very deep chamfer while the other half didn’t have a chamfer at all.

I need to find a fix for that. In the meantime, I manually sanded down the edges until I thought it’s enough so nobody would get hurt.

Then, I cleared remaining chips along the engrave paths with a toothbrush, and took the title photo above.

Quality analysis

Upon inspecting the part in more detail, I made the following observations:

Prototyping coasters made of aluminium sandwich material.

Top sheet machining complete. Cutout and engravings.

Coaster holder cutouts, bottom view

Rough edges where engravings are more deep

Less wide engravings looks more clean.

Residues and chips left in the engrave channels

Wondering why engrave quality reduces on deeper engravings

All parts look the same

This actually is good news. It means that the tools I bought from vhf this time are still sharp and can be used for future set(s) of prototypes.

It also means that my machine did not experience step losses or mechanical issues and that the workbed surface is level. This is especially important in this case as the 90° engrave cutter’s engage width highly depends on workpiece surface height. With an uneven workbed, the line width would differ visibly from part to part.

Wide engravings have poorer quality than fine ones

I’m not entirely sure about why that’s the case. I’m no metal cutting mechanic so I’m missing the professional qualification, but my guess is that this is because the engrave cutter does not have a mechanism to get rid of the chips on time, mashing it into the material with its next half-turn which creates a less smooth finish.

If that is the case, an additional smoothing run at a low material removal rate would solve this issue.

Chamfering without pre-cut delivers less smooth results

My guess is that although the symptom looks different, the root cause might be the same. I’ll try with an additional smoothing run for the other side as well.

Another alternative would be to reduce the feedrate and see what happens. Maybe a balance has to be found between manufacturing time and quality?

I need to get locators to fix workpiece offset

Obviously, I wasn’t able to reproduce part placement with help of the fence when turning the part around. Some tenths of a millimeter are enough to ruin everything.

I could do the following:

Plan a Job00 that just places locator holes in the workpiece with a diameter of 5.04mm. Make sure they are symmetrical so the workpiece can be flipped around the diagonal edge facing to the fence for two-sided milling, while the workpiece edge’s position remains unchanged.
Place the workpiece against the fence and have Job00 executed.
Remove the fence.
Get some 5mm register pins and put them into the vacuum table’s holes, matching the workpiece’s holes.

Or I could just cut through and manually add the chamfer on the lower side with help of a hand router. As the workpieces are quite small, I’d have to design a holder first to keep my hands safe. Maybe that’s even faster than the above approach…

Some chips are stuck down in the engrave path

I don’t know why that’s the case. Will have to talk to my tool manufacturer to maybe get that sorted out.

Summary

All in all, I can be pretty happy. The machine does what I want it to, and the overall result is OK already now, early in the prototyping phase. The composite material is nice to work with and won’t eat cutters as quickly as other materials (like HPL). The surface finish looks really nice and its “upmarket premium” touch adds to the flair. The chips are a bit annoying, though, because they like to to stick to everything and don’t want to be vacuumed up.

Anyways, the quality ist not fully satisfying yet so I’ll keep adding knowledge and experience to yield some improvements in the future.

Github Actions: Troubleshooting

Schallbert — Tue, 14 Feb 2023

Motivation

Why I’m writing a short post about Github Actions? Because I wanted to use it for my latest Software-project, QR-codengrave. When at some distant point in the future, I have a new computer, don’t remember which IDE I used to build and run, test, and create assets with. Or, say, I have a corrupted virtual environment or launch.json, I still want to be able to deploy that application, publish bugfixes or bump a release.

Although that’s not the usual argument for using CI/CD - typically the first reason being that collaboration on a piece of software becomes easier - still the appeal is strong enough to try Github’s automation called Github Actions.

I won’t try to run nightlys or have a continuous deployment pipeline as an end in itself, though, and only want an automated build, lint, test run, and deploy stuff when merging back to the production branch.

Got it, but why a dedicated post?

Because it was such a pain to get it right. I spent hours and hours pushing and hoping that, this time, the goddess of Github Actions would actually have my solution built without errors and at least run some of the tests.

Note: If I had been more diligent, I would have installed yet another tool like this to have my actions run locally, both saving time and the embarassment of tens of failed builds in a row. But I wasn’t.

I had quite some errors that appeared on the way of making QR-codengrave. Some I wasn’t even able to properly resolve so I had to utilize work-arounds. But let’s get to that later.

The script

Github Actions uses YAML to take orders for its pipelines. It is very well documented and provides many readily-working scripts for most scenarios and programming languages.

The so-called “Workflow file” looks like this:

# This workflow will install Python dependencies, run tests and lint with a single version of Python
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python

name: python_integrate

on:
 push:
 branches: [ "master" ]
 pull_request:
 branches: [ "master" ]

permissions:
 contents: read

jobs:
 build:

 runs-on: ubuntu-latest

 steps:
 - uses: actions/checkout@v3
 - name: Set up Python 3.10
 uses: actions/setup-python@v3
 with:
 python-version: "3.10"
 - name: Install dependencies
 run: |
 python -m pip install --upgrade pip
 pip install flake8 pytest
 pip install qrcodegen # Dependency install of qrcodegen
 if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
 - name: Lint with flake8
 run: |
 # stop the build if there are Python syntax errors or undefined names
 flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
 # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
 flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
 - name: Test with pytest
 uses: GabrielBB/xvfb-action@v1  # Diverts tkinter GUI to a virtual frame buffer (VFB)
 with:
 run: |
 pytest

And it does not divert too much from the standard template that I was using first. The only things I added are the run pytest entry and the usage of a virtual frame buffer to cirumvent issues with my GUI which makes the tests fail when the GUI tries to fire up and there’s no screen to show it on.

Github Action runner errors

The following list of fails describe the errors and resolutions that I faced until I had a stable CI at around action run #50.

Directory mismatch?

Output:

/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/importlib/__init__.py:126: in import_module
 return _bootstrap._gcd_import(name[level:], package, level)
test/test_machinify_vector.py:4: in <module>
 from bin.platform.machinify_vector import MachinifyVector, Tool, EngraveParams
E ModuleNotFoundError: No module named 'bin'

Reason: I placed my source files under /bin and made it available to a local instance of PyInstaller (the tool I chose to render my python sources into an executable under Windows). Upon upload, Github Actions couldn’t relate to the paths I chose and had that error prepared for me.

Resolution: Add an empty file with name __init__.py into the /bin folder. This will flag anything in that folder as package, thus making it available to GH actions. So what looked like a directory mismatch actually was a package-not-found error that my IDE didn’t have as it knew which files I had created.

YAML syntax errors

This error occurs because I was trying to put two workflows into one file. Somehow Github Actions seems to only accept a single one per file.

Path-not-found errors

I had a lot of those and they were painful to resolve. The folder structure that I was targeting looked like:

- assets  # images, persistence file, etc.
- src  # source files
- test  # pytest files for unit and integration testing
- dist  # build artifacts
- build  # build process files

But somehow, it was super hard to make my local IDE build environment, Pyinstaller, and the Github Actions Worker and its trigger of Pyinstaller to cooperate. One of those four parties would always complain that a path is missing or something else was wrong. That’s why I decided to abandon relative paths and instead use python’s importlib_resources. But in the end, this turned out to be problematic with Pyinstuller on remote Github Actions.

What finally solved the problem was to move assets into src. This way, the ./ command couldn’t go wrong anywhere and although I don’t like the construct very much, I was tired of putting more time into that for a clean fix (If you know how to handle path pointing in Python for both local and remote, let me know in the discussions).

Tkinter headless testing

Later in the development process, I decided to add some integration tests into my solution. This way I wanted to make sure that child windows would actually perform the callbacks to update the main application, and vice versa e.g. that the persisted tool list is cascaded into the tool configuration window.

These tests were running locally but with a disadvantage: When I simulated an error or warning case, the corresponding messageBox would insist to be closed manually by the user before continuing the tests. Which was just a bit annoying for me in the IDE to do, but would just be impossible to do from within Github Action’s CI pipeline.

So I bit the bullet and added a wrapper class to tkinter’s messageBox so I could inject a mock that wouldn’t actually trigger the popup:

class MsgBox:
 """Re-implementation due to testing purposes: With this trick, we are able to mock these windows
 so we do not have to wait for users to manually close the dialog, unblocking the application again."""
 def showinfo(self, title, message):
 showinfo(title=title, message=message)

 def error(self, title, message):
 showerror(title=title, message=message)

Although in theory it should be possible also to invoke the popup’s “OK button” from within the test, I wasn’t able to have that automated reliably.

When this worked, I pushed to Github with high expectations - and got another beautiful error:

_tkinter.TclError: no display name and no $DISPLAY environment variable

At least that was easy to understand: Tkinter just didn’t know where to draw the windows - seems legit when there’s no display connected.

So I searched the internet and found a single line of code that miraculously solved my problem by introducing a virtual frame buffer:

uses: GabrielBB/xvfb-action@v1 # Diverts tkinter GUI to a virtual frame buffer (VFB)

Now, on Push, Github Actions rained green ticks down on me which felt really good for a change.

QR-codengrave V1.1 / V1.2

Schallbert — Sun, 12 Feb 2023

In a nutshell

Faster Engrave algorithm
Several bugfixes
Usability improvements

Demo video

QR-codengrave Quick Demo↗

Version 1.1

Engrave algorithm upgrade

The algorithm now scans the lines horizontally
One line “Left-to-Right”, next line “Right-to-Left” etc. to reduce machine travel
prefers lines over dots, and longer lines over shorter ones for time optimization
draws lines vertically or horizontally
rapid move positioning now is packed into vectors that can be called to new Points(X, Y) simultaneously, not only one dimension at a time
New algorithm just needs 80% of the time for engraving compared to the old one

Bugfixes

splash image (qruwu.jpg) is not shown on startup when Persistence.dat is not available
Crash when pressing Stop Draw although drawing has never been initiated before
Crash when adding a tool on an empty selecion list
Initialization of application with no tool selected disables dropdown menu
Fix a calculation error in QR-code size estimation
Fix an error pixel width calculation for tapered tools
Actually the currently selected tool is not shown “selected” in the dropdown on startup
QR-codes can grow larger than the display area

Usability improvements

Change default tool name from name to default for clarity
Make non-tapered (V-cut) tool the default tool for better usability
Setting a new XY-zero offset became more intuitive (Y down is negative, X right is positive)

Version 1.2

Minor change over 1.1, most importantly a Bugfix that could lead to tool change macro not being called. In addition, tapered tool angles are now taken into account when calculating track width and QR-code engrave size.

Bugfixes

Tool select Tx and tool change command M06 are now in the same line of the G-code to correctly trigger tool change macros
Actual selected tool not shown on startup, instead the first tool of the list was shown
QR-code drawing could grow larger than the screen. This is fixed now.
Crash fix when trying to update status although no engrave parameters available

Usability improvements

Add application version to G-code as comment
Add qr_ prefix to G-code save-to dialog’s default name
Improve job duration estimate

Availability

The latest release is available for download here, free of charge for home and private use. For more info on the project, please find the readme and the background information linked here.

Migration from an earlier version

It is possible to copy the persistence file to the new version’s directory. This way, saved tool lists, engrave parameters, and XY0 offsets can be ported to the new version. How it’s done:

Navigate into src/assets of your QR-codengrave folder.
copy Persistence.dat to a temporary folder or e.g. your Desktop.
Remove the current installation by deleting the folder.
Download the latest release, move it where your previous release was deployed.
Move the Persistence.dat into src/assets of your new installation and start the application.

Breadboard Fireplace

Schallbert — Sat, 10 Dec 2022

Material list

Item	Count	~ Cost [€]
LED: Warm-white, e.g. Ifwd=30mA, 3lm	5	1,50
LED: Yellow, e.g. Ifwd=35mA, 1lm	2	0,60
LED: Red, e.g. Ifwd=35mA, 0.4lm	1	0,30
Arduino Nano Atmega328	1	18
Breadboard e.g. 170 pins	1	0,70
Jumper wires, pin-pin, e.g. 5cm	10	3,30

In addition: Some jumper wires, Mini USB cable, powerbank / wall adapter

Tools

optional: hot melt gun
optional: soldering iron

The Build

LED closeup for your reference ^[1]

Connect the Arduino Nano’s right header to the left hand side of the breadboard. Make sure all pins are on level. If possible, don’t touch the pins due to risk of electrostatic discharge that may damage the device.
Connect the anodes (+) of the LEDs to the board’s outputs D2 through D9. The anode most often is the LED’s pin that doesn’t carry the chip (left in this case).
Place the cathodes (-) of the LEDs to the right hand side of the board.
Place the header to the right of the LED’s cathodes.
To connect the cathodes, user jumper wires. Alternatively, solder the cathodes together right away.
Using a jumper wire, connect the “common cathode” to one of the GND pins of the microcontroller board.
The LED anodes shouldn’t be able to touch each other. You can ensure this by placing some hot melt at their legs.

Programming

The code is so simple that I can just paste it here. Copy it into your favourite IDE, program your board and you’re done!

Disclaimer: This piece of code is “quick’n’dirty” and has lots of potential for refactoring. For instance, the pin IDs are hardcoded, I’m not using C++11’s range-based loop, and the speed algo will hit the low limit quite often and not show random behavior. I’m happy to hear your suggestions on my Github discussions page.

#include <Arduino.h>

#define MAXLONG 0xFFFFFFFF
#define SPEED_HIGHLIMIT 1000 // 500Hz
#define SPEED_LOWLIMIT 0x1FFF // 20Hz

void calcFlicker(uint32_t randomFlicker);
void writeLed(uint8_t columnCode, uint16_t speed);
uint32_t iRNG = 1;

void setup() 
{
 randomSeed(analogRead(0));
 pinMode(2, OUTPUT);
 pinMode(3, OUTPUT);
 pinMode(4, OUTPUT);
 pinMode(5, OUTPUT);
 pinMode(6, OUTPUT);
 pinMode(7, OUTPUT);
 pinMode(8, OUTPUT);
 pinMode(9, OUTPUT);
 iRNG = 1;
}

void loop() 
{
 iRNG = (iRNG >> 1) ^ (-(iRNG & 1u) & 0xd0000001u);
 calcFlicker(iRNG);
}

/*
 This function takes a 32bit random value,
 and moves this value along a window of an 8bit 
 flicker output value.
*/
void calcFlicker(uint32_t randomFlicker)
{
 uint8_t currentFlicker = (uint8_t)(randomFlicker & 0xFF);
 uint16_t randomSpeed = min(SPEED_LOWLIMIT, (uint16_t)(randomFlicker & SPEED_HIGHLIMIT));
 for(uint8_t i = 0; i < 24; ++i) // 24 because of 32bit length of random, may not run empty
 {
 writeLed(currentFlicker, randomSpeed);
 randomFlicker = randomFlicker >> 1;
 } 
}

/*
 this function outputs a code to an LED column of 8 LEDs
 connected to arduino's pins with a selectable delay.
*/
void writeLed(uint8_t columnCode, uint16_t speed)
{
 digitalWrite(2, columnCode & 0x01);
 digitalWrite(3, columnCode & 0x02);
 digitalWrite(4, columnCode & 0x04);
 digitalWrite(5, columnCode & 0x08);
 digitalWrite(6, columnCode & 0x10);
 digitalWrite(7, columnCode & 0x20);
 digitalWrite(8, columnCode & 0x40);
 digitalWrite(9, columnCode & 0x80);
 delayMicroseconds(speed);
}

Working

Once you power the device up via USB, the LED flicker sequence should start and all LEDs should light up eventually. You’ll notice how the glow calms you down. Maybe you have some twigs ore firewood at the ready and an empty space in your shelf? Take a powerbank and hide your Breadboard Fireplace below its wooden decoration. You’re done!

Back to Project main page

Vacuum Pump

Schallbert — Fri, 28 Oct 2022

What is it?

A vacuum pump is an appliance that is able to evacuate a vessel, i.e., to reduce the amount of gas atoms within a container. It does not seem to be possible to shrink the number of atoms within to 0, but of course this is by far not required for our application in which a so-called technical coarse vacuum is sufficient.

There are different types of vacuum pumps with specific properties that I’ll try to shed a light on later.

How does it clamp workpieces?

For clamping with the use of a vacuum table (see my related post), we’re only interested in the pressure difference the vacuum pump generates versus ambient pressure which is (103000Pa = 10.3N/cm² = 1kg/cm²). This is the maximum clamping force a vacuum table can generate in theory.

Pump capacity vs. vacuum pressure

The key parameters of any vacuum pump are the minimum absolute pressure they are able to draw in hPa (at low or even zero throughput) and the volume of air they’re able to put through within a given time in m³/h (at low or zero pressure difference).

This is comparable to the electrical world, where open circuit voltage (pressure difference) and short circuit current (throughput) e.g. in photovoltaic cells are given parameters to calculate an optimal point of operation.

Here, you can add more cells to either get more current or more voltage from the system by connecting them together in parallel or in series. For vacuum pumps, it’s often more practical to choose a more powerful motor that drives the pump than using series/parallel operation. But still, there’s a trade-off to make. Certain pump types are very well suited to generate lots of throughput, but then are lacking vacuum pressure and vice versa, pumps that have very good vacuum ratings often have considerably less throughput than other pump types with the same power level.

If you want both, prepare to spend a lot of money. So again, it all comes down to what you need for your application.

Pump characteristics

Max. Volume and max. pressure difference can be entered as data points in a XY-diagram, and a line can be drawn between these two points which gives an approximation of overall pump behavior. While the position of these points largely depend on the pump’s motor power, the line’s slope is determined by pump type and design.

Below find a comparison of these different pump types. For this example, I arbitrarily chose a motor power of 1.5kW and selected some pumps with that rating of which I could find datasheets online.

Of course this approximation is very rough and holds true for high-throughput pumps better than for deep vacuum pump types. For the latter, the function looks more like a hyperbolic cosecant (see image below) than a linear graph due to their lower tolerance for leakage air.

We can use the pump’s characteristic curve the following way: When we have a given air volume demanded by our application (like a vacuum table), we can read the reachable pressure difference for this specific pump. This way, you can pre-select a pump type depending on your requirements.

Pump types

There are a multitude of pump types in the industry. They have different principles of operation and - of course - are suited for different applications. I’ll concentrate on the four types that are most often used for machining.

Radial blower (High Throughput)
Sidechannel compressor (High Throughput)
Piston pump (Deep vacuum)
rotary vane pump (Deep vacuum)

Radial blower

Radial blowers accelerate air between their blades. Air enters the blower at the center of the fan which creates a lower pressure here than in the surrounding air. Then, it is pressed towards the outside due to centrifugal forces as the fan rotates, and is exhausted at the perimeter.

Radial blowers are used for many applications like vacuum cleaners, inflatable castle blowers, forced air delivery for combustion processes, ventilation systems etc. They are simple, come cheap, and are well suited for applications that require very high throughput. The downsides are high noise levels and low pressure differences relative to most other compressor types. However, if you’re working with large surface sheets or porous material, the radial blower e.g. of a forced-air cooled vacuum cleaner motor might be just the right thing for your application.

Sidechannel compressor

Sidechannel compressors combine high capacity with medium pressure difference. They are built single or multi-stage (series circuit) and thus can favour higher capacity or pressure difference according to your needs. Like radial blowers, they interact with the medium only and thus do not have part to part friction that would require frequent maintenance. Like radial blowers, they tolerate vapours and can even be used in dusty environments.

If you want the sidechannel compressor to provide high pressure differences, you’ll have to buy very powerful (thus expensive) drives as their characteristic curves are shallow. Their noise can be efficiently muffled, but they often carry a higher price tag than comparable radial blowers.

In this list, they are “the jack of all trades”. The vacuum they provide can be upgraded to high-enough levels for most applications that don’t require extremely strong clamping and their price is moderate. Cost of operation is determined by the electrical power they consume.

I found this in-depth description of the sidechannel compressor helpful: GutmbH.

Piston pump

Piston pumps are commonly used in compressors and vacuum pumps alike. They provide deep vacuum levels at a low capacity, don’t use much space in your shop, and are reasonably priced.

On the other hand, their output stream is pulsating and they tend to be noisy. They also cannot handle liquids or vapours and have higher requirements on maintenance than frictionless pumps like sidechannel compressors or radial blowers. I did not find so many notes about their usage for vacuum tables, but that does not mean they’re not suited for this specific application.

Rotary vane pump

Rotary vane pumps come in two classes again: Dry-running and oil-lubricated. While dry-running pumps have similar pressure data at higher capacity than piston pumps with the same power rating, oil-lubricated rotary vane pumps can provide a much deeper vacuum level and even higher capacity with the same power rating. On the downside, they commonly need oil warmup and de-gasing cooldown runs, and have a higher maintenance cost due to their oil system.

Their principle of operation: A motor drives a slotted dish mounted excentrically within the vacuum chamber. In the slots, vanes are placed that can move freely. As the pump’s motor accelerates, the vanes are pressed against the vacuum chamber’s walls, sealing it against the other cavities. This way, compressible media can be transported only.

Rotary vane pumps deliver a continuous flow at low noise levels. The vanes are expendable parts that need replacement from time to time, which is especially true for the dry-running specimen. If you just compare capacity, they usually cost twice to three times as much as sidechannel compressors.

You should get one of theese if you often mill small workpieces at high lateral forces, e.g. mild steel. In this scenario, you can even keep your costs at bay when you’re not cutting through as in that case, leakage air is no issue of yours so you can choose a very low pump capacity.

If you’d like more detail, please find this well-made page (in German) from GutmbH.

Let the workpiece determine your pump

OK, so you chose the type and size of vacuum table you require for your jobs. Great! Now all you still need is a vacuum source and a way to feed the vacuum into your table.

As you can see in the figure, the downforce you get highly depends on the surface area of your workpiece. The smaller your workpiece is, the more pressure difference you require to maintain the same force - with the atmospheric pressure being one limit and the absolute vacuum the other. This makes vacuum tables both a very homogenuous but “weak” clamping method.

Now, what you actually want is to protect your workpiece both from being lifted off the of the machine bed by the tool, at the same time avoiding parallel shifts and rotation of your workpiece while machining. So we have to calculate the downfore created. In a second step, we want to convert that downforce into lateral forces that have to be higher than the maximum lateral force applied by your tool so your workpiece keeps its position.

Downforce

The clamping force to your machine bed is easy to calculate:

$$f_{clamp}= F_N = \Delta p A + F_g $$

It is measured in Newton and determined by the pressure difference at the workpiece top/bottom its surface and its weight that gravity forces downwards.

The value you require heavily depends on tool size and geometry, workpiece material, and feed/speed. You’ll require a lot more downforce for a scenario in which you’re milling small workpieces of sturdy materials like HPL with large tool spiral angles and tool diameters at high feeds and low spindle RPM than for cutting plywood sheets with small-diameter straight flute bits at high RPM and low feed rates.

Lateral force

To do the translation of downforce into lateral forces, we’ll introduce the coefficient of static friction $µ_s$.

Let’s assume you use a sheet of rubber between a metal workpiece and your vacuum table. We further assume a coefficient of static friction µs = 0.7 that I found online which seems really conservative for multiple reasons: First, it doesn’t take surface roughness of both rubber and metal into account, and, more importantly, we’re pressing the workpiece into the rubber which yields a form-fit bonding that increases breakaway force a lot.

Anyways, all you need to do here is to take your downforce calculated earlier and multiply it with this factor:

$$F_s = F_N µ_s$$

As you can see, there are two things you can manipulate with the same linear weighing: The material pairing of workpiece surface and machine table determine the coefficient, and the downforce created by your pump.

Rotation

This calculation becomes a bit more tricky. Interestingly, you’ll require less force to rotate a workpiece out of position than to parallel-shift it away. This is why:

When you rotate, say, your mobile phone on your desk around its center (touching it at its top-right corner), not all imaginable points on that phone have to move the same distance. Points closer to the center of rotation will have to move a shorter distance for the same angle of rotation than points at the tip of the phone. For a parallel shift, however, every imaginary point had to move the same distance.

This pheonomenon seems independent of the object’s shape once you compare travel of the outermost point of that object in relation to its center of rotation.

For vacuum clamping, worst-case workpiece and job combination is long, sleek workpieces that need a slot to be milled orthogonally to its longest dimension at the outer edge, creating the highest torque that might lead to rotation combined with little surface available for vacuum clamping.

Best-case of course are round or square workpieces where you cut close to their geometrical center.

Which pump should you buy?

Ok, let’s sum it up:

It’s always a combination of workpiece material (easy/hard to mill, low/high static coefficient of friction), minimum workpiece size (the bigger, the easier it becomes to generate high clamping forces), leakage air tolerance (porous surfaces, cutting all the way through), vacuum table type and size, and milling parameters that determine which pump to buy.

You should collect some experience before buying a vacuum clamping solution: What is your typical workpiece size and material? Do you often do cutouts? Do you require quick execution that exposes your parts to high forces?

Some recommended combinations

Aluminium, very small workpieces, not cutting through: Low capacity piston or rotary vane pump, grid vacuum table
Wood and MDF, large workpieces, cutting through a lot: radial blower or single stage sidechain compressor, porous surface table
HPL, small workpieces, cutting through: High capacity rotary vane pump or dual stage sidechain compressor, hole grid table, optimized CAM strategies for “late cut-throughs” at low depth increment

What pump I chose

I ordered a relatively large table (730 x 655 mm) with 0.5mm holes arranged in a 10mm grid. As I’m still testing a lot of different materials for prototyping purposes, my workpieces never look alike or have similar sizes. That’s why I had to swallow the pill of spending a lot of money on a pretty capable pump that would both do high throughput for bigger wooden sheets that I’d create cut-throughs in and deep vacuum levels for small aluminium workpieces.

Rotary vane pump comparison (Image courtesy of Stritzelberger GmbH) ^[1]

The pump I chose is a dry-running vane-type vacuum pump with a capacity of 40m³/h and a maximum pressure difference of 880mBar. Its air leakage vs. pressure diagram should look similar to the marked characteristic in the plot above. It weighs approximately 40kg and uses a 3-phase induction motor with a nominal power of 1.3kW.

I am using a version where the motor is operated on a single-phase with help of a capacitor due to the fact that I do not have a three-phase outlet available. I also added a motor circuit breaker to help protect the machine against overload and overcurrent.

Is it loud?

It depends on how you define loud, of course. You can compare its 68dB(A) noise level - measured at a distance of 1 meter - with a typical vacuum cleaner, just that it sounds pretty different. When the CNC machine operates, e.g. running through material with a fairly big endmill, it is still quiet and you won’t realize the pump is on.

I even made a short video for you to view and hear for yourself:

Rotary Vane Vacuum Pump - demonstration↗

CNC Vacuum Table

Schallbert — Fri, 21 Oct 2022

Let’s assume you clicked this article on purpose and didn’t just “surf the internet”. So you already know what a vacuum table is (If not, continue reading here). Its principle of operation is depicted in the figure for your reference. You probably also know there are multiple types of tables serving different purposes. And you maybe even know that you want one.

Great! This article both describes my journey towards acquiring this piece of tech and also will try to help you select the right one. Let’s start with a little interview.

Why do you want a vacuum table?

This is the most important question.

🤩 Do you want it because it’s expensive and looks great?
😪 Or because you’re tired of re-adjusting clamps?
😖 Or because you broke a lot of endmills running through the former by accident?
😨 Or rather because you experienced that workpieces tend to learn how to fly when the adhesive force of your tape is just not high enough?
😎 Perhaps because you`re going professional and you want to cut preparation and production time?
😕 Or maybe because the work results are not as good as they could (or should) be?

My Reasons: Quality and Speed

I’d go for the latter. I have the following issues I hope a vacuum table can fix:

When engraving coated materials (Dibond, HPL etc.), slight Z0 surface height differences ruin the work, especially on bigger workpieces
Cutouts from a bigger sheet have to have taps to keep them from slinging away. These taps are cut away later but are still visible on the end product.
When I don’t want to use taps, I have to leave an onion skin at the lower layer. With wood, this skin has to be thicker and I need a second job for chamfering which adds time expenditure.
Some sheets (especially wood) are concave. Thus, when trying to go all the way through, some parts are fine and others are not fully cut.
My machine suffers from vibrations, especially when using 2-flute wood roughers and high Z+ depth per pass at aggressive feeds. This might be because I use just a couple of clamps to save time and to keep travel paths clear.

Which type of table do you need?

Ok, you know why you want one, now we should find out which type is the correct one for your jobs. Note that vacuum tables are most suitable when you’re milling not too flexible sheet material as the table’s clamping force depends on the surface that is in touch with it.

There are three families of vacuum tables:

Grid tables
Hole grid tables
Porous surface tables

Grid tables

A grid vacuum table (Image courtesy of Stritzelberger GmbH) ^[1]

Grid tables have a single (and often centrally placed) suction hole. Their vaccuum area is defined by a (foam) rubber band that can be placed on the grid within the outline of the workpiece. As the rubber band is compressed by the workpiece’s downforce created by the vacuum, the seal is airtight and just very few leakage air has to be evacuated. That’s why they have low requirements on the vacuum pump’s required capacity. They can also be used to clamp materials with rough surfaces due to the rubber band, e.g. saw-cut wood. Grid tables’ designs are often simple and thus affordable.

Altough they really shine here, there are two possibly vital disadvantages: First, when you’re doing singletons, you’ll have to place the rubber band anew for each workpiece. So your XY Zero changes from part to part as most often the workpiece has to be placed centrally on the table. More sophisticated grid tables may have multiple suction holes, and the active one can be selected by the user. They may even offer fences and excenter stops to reduce set-up time. Still, you might not be able to cut any form you like as your limit is the grid.

Second and more important, you run into issues once your part has to be cut “through”. This will immediately make the vacuum collapse and the pump will be overwhelmed with leakage unless you take precautions (build adapter plates or underlay fences etc.)

In short:

Grid tables work very well if you don’t cut through your workpieces.
Grid tables have lower requirements on the vacuum pump and can even hold parts with rough or slightly uneven surfaces.
Grid tables are cost effective but may require additional care when zeroing in workpieces with different dimensions.

Hole grid tables

A hole grid vacuum table (Image courtesy of Stritzelberger GmbH) ^[2]

Hole grid tables have - as the name suggests - blind holes that are aligned on a grid. Within these holes, there’s a tiny suction hole that’s size matches the material’s properties you’re machining predominantly - tiny little holes for smooth or polished surfaces like face milled metal, and still tiny but bigger ones e.g. for hard wood and other materials that may have higher air leakages. They often have switchable vacuum chambers below these holes so that the area of where the vacuum is applied can be changed. Uncovered area around the workpiece can be covered with a foil to raise vacuum forces and efficiency. These tables often consist of multiple parts, e.g. a bottom layer for mounting to the machine bed, a middle layer to embed evacuation ducts and the top layer carrying the hole grid.

These tables can clamp workpiece of any shape that have flat bottom surfaces. When your workpieces are really small, of course, the clamping force will diminish - that’s physics. The small suction hole allows for through-cuts: Affected holes do loose their capability to hold the workpiece in place on one hand; but nearby holes on the grid are merely affected as air leakage is limited by the “lost” suction hole’s diameter even at high pressure differences. Aligning the workpiece is even simpler with hole grid tables as their grid’s holes can be used to hold locator pins that build a fence.

Their disadvantages: These tables tend to be more expensive as their design is more complex. They often have multiple vacuum circuits and thus require more connections to your vacuum infrastructure (that have to be kept leak-free). Their tolerance of leakage air and lateral inflow means that your vacuum pump requires to have a higher capacity.

In short:

Hole grid tables support any sheet shape as long as its surface is flat and big enough to meet your clamping requirements.
Hole grid tables tolerate through-cuts at the cost of higher requirements on pump capacity.
Hole grid tables make workpiece alignment a cake walk.
Hole grid tables are complex and often expensive.

Porous surface tables

A Porous surface table (Image courtesy of 'Jens´ Vakuumtische') ^[3]

These special kind of tables do not need holes or channels on their surface at all - their top surface, instead, is air-permeable. I’ve seen specimen where the top layer is made of MDF so actually it’s not a big problem if you accidentally got Z0 wrong - just place a new sheet of MDF on top and off you go again.

The tables are lightweight and - with a lot of knowledge about materials science and thin-wall milling - relatively easy to construct as a DIY project. There are many examples out there on the internet. They work best with a high-volume blower and thus often do not support large pressure differences - so as long as your workpieces are not tiny, that’s not an issue. The tables are suitable for cutting all the way through as well. They are cheap to buy and to upgrade with additional features, and a vacuum cleaner with forced cooling might already be enough to power it up.

The downsides: Might be subject to vibrations due to their low weight, best not to be operated with cooling liquids, unfavorable static friction coefficient with most pairing materials due to MDF (thus more surface needed to protect from slipping).

In short:

Porous surface tables are cheaper to have and to operate.
Porous surface tables are well suited for big workpieces and materials that don’t require cooling liquids.
Porous surface tables may not be perfect for high precision milling.

What all these types have in common

Common to all these types is, that the downforce can safely be calculated the same way for a given vacuum pressure applied to your workpiece (not necessarily identical with what you see on your gauge!). No matter if you have a grid, a hole grid table or a porous surface table, you can always calculate the full surface of your workpiece (and not just the sum of holes’ area within the workpiece’s outer bounds). This is because microscopic surface irregularities of both table and workpiece material allow a lateral airflow into the suction area of your table, making pressure differences between atmosphere and vacuum pump side homogenous.

On the other hand, this is also true for your workpiece’s outline, so there will always be a little leakage air in your system with the side effect that clamping force deteriorates the more you approach the peripheral zone.

That said, cutting all the way through your material will create extra peripheral zones that can, when situated too close to each other, even have zero clamping force left.

What else do you need?

Well, there’s a couple of things you do require. A vacuum pump, for instance, that supplies your table. But you’re in luck, I wrote an entire post on that topic as well. But there’s more, actually:

Fasteners, washers, nuts to attach the table to your machine bed
A vacuum manifold that may allow you to select chambers within your table, preferably providing a vacuum gauge to check pressure levels
Vacuum hose(s) to connect the table to your manifold
Another vacuum hose that connects the manifold to your pump
A pump motor switch and motor circuit breaker
Vacuum fleece or rubber mat (hole grid tables only)
Vacuum gasket thread (grid tables only)
porous spoilboard (porous tables only)

Conclusion

All in all, it will cost you a lot of your time and money to get and setup a vacuum table. So you should have a couple of good arguments to go through that effort. If so, chances are high you’ll be rewarded with quicker execution, better quality, and shorter job setup times.

My selection

I exclusively work with sheet material. I process wood, high pressure laminates, acrylic, MDF boards, and often have to cut out parts from a bigger sheet or use nesting for many different parts. This is why a grid table wouldn’t work for me very well.

Due to the vibration issues I have with my current setup lacking enough weight and the fact that a ready-made vacuum table didn’t exist for the dimensions of my machine, I didn’t choose a porous surface table either. This would have been the most cost-effective solution, though.

Instead, I contacted a vacuum table manufacturer to buy a couple of small hole-grid tables that I could individually switch on and off depending on workpiece size and pressure that I’d require for secure clamping. Well. I was told that a much simpler and better solution would be to buy one of their even heavier aluminium tables. Less worrying to get multiple tables supplied with airtight suction ducts, valves, connectors. Not having to access both sides of the milling machine with ducts. Not having to adjust the tables to sit together tightly or face mill for a level surface. And by the way, they’d create a customized design so it would perfectly fit my machine…

What can I say. I scraped money together for this much more elegant solution. I made a drawing of my machine bed. Just a couple of days later I got a sketch back with an initial design (this one is still missing additional holes for clamping vises, and the grid is less dense to keep processing speed high). I gave my OK for production.

Table delivered

The manufacturer is located not too far from where I’m at home. So I went there and collected it myself. When I had unmounted my old spoilboard, I thought it would be a good idea to have some heavy foam acoustic absorption panels placed in the machine frame to dampen noise and resonation from below the table.

Only afterwards, I bolted the new vacuum table to my machine with help of M6 fasteners and hammernuts. After that it took me multiple hours to get a more or less even surface. As the machine frame’s tolerance is relatively high, I initially had deviations of 3/10mm over the vacuum table’s surface 😓. With the help of some paper and thin cardbord, I was able to smoothen out the roughest dips, but I could still measure a deviation of 15/100mm.

CNC Z-axis bed leveling check↗

To get to an optimal solution, there would be three options:

Surface milling of the whole table
Unmounting the table, adding thin plastic foil where needed, remounting, adding additional foil/paper where needed…
Unmounting the table, loosen the machine frame and get to the root: the frame’s sides are rotated towards the inner just a tiny bit, and the front cross-beam seems to be bent upwards just a little, so it should be lowered by less than 1/10mm.

For the time being, any of these options seemed too much work as I don’t know yet if I need it to be that exact. So I continued by connecting the vacuum hoses.

Connecting the manifold

While happily cutting the hoses that would connect the vacuum table to the manifold, I thought “Wait a second - wouldn’t it be better to have the 9mm hoses as short as possible and rather use a longer 19mm pump connector hose for less flow resistance?” I stopped work and did some calculations:

My table has ~4900 holes of 0.5mm in diameter. Its combined cross section is $A_{table}=962mm^2$, which is equivalent to a hole with 35mm in diameter (differences in flow resistance ignored).

The seven connecting hoses have a diameter of 9mm. Together, their surface is just $A_{hose}=445mm^2$, equivalent single diameter of 24mm which is less than half than the table’s flow section.

For the pump hose and its diameter of 18mm it gets even worse: $A_{pump}=254mm$ is less than 30% of the table’s capacity.

So when I assume an even air leakage over the seven chambers of my table, it makes most sense to have the pump hose as short as possible as the flow speed is highest here and so are the losses. For a scenario where just one of the table’s section is active and there’s a lot of leakage, obviously the 9mm hose should be as short as possible.

I decided instead to never allow full leakage so I could keep my preferred, comfortable location of where the manifold is mounted and just completed the wiring. 😉 By the way, the first part I manufactured on the new table was the holder for the manifold you can see in the image below.

Estlcam: Negative Carves

Schallbert — Tue, 27 Sep 2022

I planned to have a wooden sign carved for a colleague of mine. But I wanted it to be inverted, i.e. the letters should stand from of the background. As there are multiple ways to accomplish that in Estlcam (a Computer Aided Machining tool) but not all are quick or easy to find, I’ll post a quick step-by-step guide here.

1. Import your frame

Open a drawing, e.g. a DXF of the frame of your sign. I designed one with rounded corners and an outer border in CAD before. If you’re carving in wood, make sure that your carve depth is not higher than the width of your frame - otherwise the frame might not have enough stability.

2. Design your text

Add the text you want the sign to “wear”. Update the text field width so it fits your sign. Then select the text in your text field and choose a font. Finally, choose a text height. Depending on the cutter you’re planning to use, select the carve depth. Cutters with sharper angles may have deeper carves than e.g. the classic 90° V-cutter. For my 12mm sheet, I chose a carve depth of 4.5mm. Then specify a maximum carve width (how big the “valleys” in your sign should be). I wanted a flat valley, so I entered a large value here. Last but not least, press the select frame button to choose have the limits of your carve defined as the inner bound of your frame.

Leave the window open, please.

3. Tool selection

Select the tool you want to do the carve with by klicking the appropriate line in your tool table. Do not yet select the disassemble Text object checkbox yet so you can still make adjustments. Click OK to close the window. Hit Preview to get a feeling for how the carve will look like. See the parallel lines, sitting tightly together? This reflects the parameter clearing feed of the cutter in the details view of the tool table. Make sure that it’s at or below 5% for a not too coarse result.

Anyways, you will not get optimal results yet because there’s just one tool for the whole job although the “valleys” would much better be cut with a standard cutter that has a flat face.

4. Clearing

Save your project now - you cannot edit the text or its carve parameters anymore after the next step. To get the clearing feed cutter in place, select the carve toolpath again and tick the box disassemble Text object. After pressing OK again, click the toolpath. You’ll now see the tool config for the carve. Add a clearing tool. Ideally, its diameter is smaller than the distance between letters and frame so you can keep the valley’s surface as smooth as possible (Note: I didn’t do that so there are some residual rough spots on my example workpiece). Maybe, also reduce clearing feed of that cutter for this job so the surface quality will become even better - values of 20-40% return very good results.

5. Preview Number two

OK, now you should be all set. Check paths and tool changes in the preview. Look how your manufacturing time decreased as well 😉. One more tip: Under Settings, reduce the safety height of your cutter to a minimum while making sure you don’t crash into clamps. Don’t worry if your sign is not legible anymore in the preview. That’s normal as it just represents the tool paths.

Re-check your feeds and speeds for the tools you’ll be using. When your carves are not very deep, there’s probably headroom for increased values here.

6. Hit the CNC

Export the g-code file from Estlcam and have your CNC build your inverse carve sign. I hope you’re happy with the results!

AnywhereAmps Alpha

Schallbert — Sat, 10 Sep 2022

Plans

{% include gallery id=“gallery” caption=“Plans for AnywhereAmps Alpha. Click to enlarge.” %}

Lists

Tools

Tools needed
Power drill
Router, compasses mount
Multi-tool: Sanding, Wood cut, cutter head
hand-held or table circular saw with guard rail
Hand saw, fine
hole cutter d>=100mm
Chisel, 12mm
Hammer
Screwdrivers, bits
Wood drills: 3mm, 4, 5, 6, 8, 10, 12mm, counterbore 45°, Forstner 30mm
Steel drills: 4.3mm, counterbore 45°
Cutter heads: 8mm straight, 12mm straight, 45°/d12mm optional
Soldering iron, solder, shrinking tube, hot melt, 2m of 0.75mm² cable etc.

Material list

Item	Size [mm]	Count	~ Cost [€]
Shell: Poplar Multiplex	12" x 6" x 7mm	1	62
Top/Bottom: Poplar Multiplex	12" x 12mm	2	8
Speaker adapter plates: Poplar Multiplex	L 300 B 150 W 12	1	2
Amp holder plate: Poplar Multiplex	L 160 B 95 W 12	1	1
Strengthening & Mounts: Birch Multiplex	L 220 B 30 W 12	4	2
Screws (preferably Torx)	12 x 3	20	2
Screws (Battery holder, handlebar)	16 x 3	12	2
Screws & nuts, PCB holders, washers	15 x M4	8	2
Foam Seal	2000 x 9 x 3	1	6
1/4" Input Jack	50 x 12	1	3
Power switch with LED, 250VAC 4A		1	3
Potentiometer 250k log 6mm axis + Volume knob	22mm diameter	1	5
9V Output Barrel Jack	5.6 brl, 2.4pin	1	2
Fan grill (protective speaker cover)	120 x 120	1	1
“I-Type” Foam Seal (Speaker)	600 x 9 x 2	1	2
Aluminium Enclosure	143 x 72 x 43	1	4
Class-D power amp board, e.g. TPA 3110	70 x 100 + 22	1	16
Piezo Tweeter, e.g. Monacor MPT-005	85 x 85 x 72	2	12
4" Woofer/Mid speaker, e.g. Visaton KT100 V 4Ohm	99 x 99 x 52	1	25
Battery, 10V - 18V	What you prefer	1	?
Electronics parts for Circuit Board, fuse…		1	10
Spare wood e.g. as mask, template, helper	e.g. 300 x 300	1	0

Electronics parts on the circuit board

ID	Item	Type	Value
R1	Resistor	0207	100k
C1	Elko	RM=2 D=5	1 µF
C2	Elko	RM=2 D=5	4.7 µF
R3	Resistor	0207	10k
R2	Resistor	0207	100k
-	Header Male	2-pin	-
-	Header Male	4-pin	-
-	Header Male	2-pin	-
C1	Elko	RM=2 D=5	2.2 µF
R4	Resistor	0207	10k
IC1	NE5532	DIP-8	-

Jumper wires

From	To	length [mm]
(11/1)	(11/2)	2.54
(10/3)	(10/1)	5.08
(14/6)	(14/2)	10.6
(8/1)	(8/6)	12.7
(9/2)	(9/5)	7.62
(3/2)	(3/0)	5.08
(22/0)	(22/1)	2.54

hole (x/y)

The Build

Cut holes

Warning When using the hole cutter in a hand tool, be very careful - I think it is one of the more dangerous tools. It is best applied gently with very low pressure and relatively high speed of >300rpm. Use a torque support and your drill’s torque limiter. Make sure the cutter’s heads are properly fastened and the workpiece is very well secured. Prepare for sudden kickbacks and mind the high intertia torque of the heavy tool.

Prepare the board for the speaker adapter plates as shown in “Plans”
Prepare the shell: Mark the middle of its height and draw a helping line on the outside. You might want to use a scale tape to account for the curvature. Then mark the three hole centers. I propose a 125mm geodesic curve between them.
Prepare the top circle for the controls panel cutout.
Choose a hole diameter of 75mm on your hole cutter.
Cut two holes for the tweeters into the adapter board. If you flip the board when you’re half through, the quality of the hole’s edges will be weigh better.
Now cut the two holes into the shell. Be very careful here, you can easily kill the inner surface if you cut at too low speeds or apply any vertical pressure.
Cut one hole in the cabinet’s top panel that you prepared. I used 76mm but nevermind, keep the 75mm.
Change the hole diameter on your cutter to 92mm and repeat for the Low/Mid speaker.
Repeat this cut as the shell’s center hole.

Routing

Put your shell onto the board you’re going to cut top and bottom of the cabinet from. Transfer its inner and outer diameters onto the board.
Find the center of this circle, e.g. by using the method 3 of this guide.
Mount the compasses tool onto your router, use a 12mm straight cutter head.
Cut the “inner” circle for the notch. I’d propose a depth of 4mm.
(optional) Mount the 8mm cutter head.
Cut the “outer” circle in a couple of steps until you have the full depth of your board cut.
Turn the circle around, and with e.g. a radius cutter, chamfer the edges.

Carving

Both the adapter plates and the battery mount sit at the shell’s inner/outer diameter, so they need to be curved to be a perfect fit. With hand tools, getting this right is kind of a challenge.

Measure the cutting depth needed so that when connecting the cuts, a curved surface will be created.
For the adapter plates, take the inner shell diameter, for the battery mount, use the outer. Note that the speaker adapter plates are convex while the battery holder has to be concave.
Use your circular saw to make these parallel cuts. Mind the blade’s width.
With a chisel, connect the cuts. Try to work along the grain of the different layers.
Use a belt sander, your multi tool, or a rasp to smooth out the curvature until it’s a good fit.

6. Atach the adapter plates on the inside of the shell using wood glue. 7. Make sure alignment of the holes is good, then take some hot melt to fix and seal the adapters while the glue is setting. 8. After some hours, either smooth out if the overlap is not perfect or take the 45° cutter head and create a chamfer. I found this to be the least controllable bit of the whole build because the router wouldn’t sit properly on the curved surface, creating rippled edges. I had to do quite some rework with the multitool.

Additional wood works

Copy the amp carrier Board’s dimensions from the plans section. Use a Forstner drill or the hole cutter to mill the ditch for the potentiometer. A depth of 6mm should be enough to properly mount the potentiometer. For your switch, a countersink might be good to get the diameter just right so that it fits in snugly. Finally, cut the holes for the jacks.

As there are a lot of battery types around, I didn’t create a specific design for the slider and battery contacts. I just used the typical crimp connectors and glued them in the adapter plate so they can establish a good electrical contact to battery plus and minus. As my battery does not have the latch included, I had the slide gradually increase thickness like a wedge so that the battery would “stick” at the correct place.

I cut some wooden rods to keep top and bottom of the cabinet closed, and at the same time protecting the wooden surface from bumps. Their length is 25mm more than the total height of the cabinet, and their width is 25mm respective 12mm at the cabinet sides.

Electronics

Cut and drill the circuit board as described in the Plans section. It is a good idea to print the plan at scale so you can use it as a mask.
Place the conductor breaks. You can use a 3mm metal drill and hand-cut them if you have no extra tool to do this.
Craft the jumper wires and place them. Use the jumper list as reference.
Place the resistors.
Use some tape to fix the parts at the circuit board, flip it, and start soldering.
Place the capacitors. Mind that you might not have enough space in your housing so that you want to have them mounted horizontally.
Now place the other components and repeat fixing and soldering them.
Prepare all connections and wires to the board: Potentiometer, source in, power amp out, VCC/GND
Check if the power supply section is working. When connecting only VCC/GND, it should draw between 2mA and 10mA of current depending on potentiometer position and supply voltage. Make sure to avoid reverse polarity, the NE5532 is killed easily (trust me, I know!). You should measure VCC/2 when you probe the virtual ground Op-amp stage’s output.
Check the gain stage. If you have an oscilloscope and a function generator, it should be an easy thing to do. If you don’t, connect the input to an audio source, e.g. your smartphone and the output to a 2k2 resistor, in series to an LED and a diode like 1N4148 or 1N4007. Turn your smartphone to full volume. The LED should start flickering when turning the resistance of the potentiometer up. At which position flickering starts depends on the LED’s forward bias, your supply voltage and your source’s maximum output voltage.
I propose to mount all the electronics and the audio input - except the 9V supply regulator - in a metal enclosure to reduce electromagnetic interference. Depending on your power amp’s wattage, you might need passive or even active radiators. For me, some holes in the enclosure will do fine.
Mount the enclosure to the amp adapter plate.
Cut holes in the enclosure as indicated on the adapter plate.
Insert switch, potentiometer and jacks. Solder the connecting wires.
Add a fuse, e.g. 4.0A to the setup on the negative path to battery for added safety.
Mount preamp and power amp and connect them properly.
Test it! E.g. use an old speaker of yours. Without any source connected, you should hear only so much as a little hissing from the speaker at full gain. When you connect your smartphone, start with gain at 0 and low smartphone volume. Then increase the preamp gain to check if the setup works fine.

Putting it all together

Mount the speakers. Use hot melt or a foam/rubber seal to make the connections airtight
Use hot melt to close any holes left, e.g. for the screws.
Solder the tweeters in parallel to the Bass speaker and connect the output cable from your amplifier.
Mount the battery holder and connect its wires to the amp’s terminals.
Put the seal around top and bottom plate so that no air can escape when mounted.
Flip the assembly and put the “top” plate with the amplifier on it in its place like shown on the picture.
Make sure that none of the speaker terminals touches the amplifier housing, risk of a short-circuiting the power amp.
The simulation shows that the Bass reproduction will improve if you add damping wool. I slayed an old cushion to get some.
Close the cabinet and fix it with the wooden stands you prepared earlier. Or use screws instead.
You’re all set (for now)! You earn an applause! Doesn’t it look gorgeous? But sure you’ll want to give it a try…

Fine tuning

Too much treble? Check how much higher the sensitivity ratings of your tweeters are compared to the Bass chassis. If it’s not on par, solder 1W resistors in series to the tweeters. I have ratings of 96dB@1W/1m vs 80dB@1W/1m according to the chassis’ technical data. So I’d have to get the tweeters a lot quieter. I tried with an 1k potentiometer and finally selected a resistance of 470Ohm which sounded just right.
The Bass flutters at high volume? Most likely, your cabinet is not airtight. Put on a test tone, e.g. 35HZ youtube link, and check if you hear or feel where the air is coming through. Fix the leak(s).
Your 9V switching buck converter emits noise into your amplifier setup? Try smoothing it with a 100nF capacitor, and maybe add a 10k parallel resistor to create a dummy load. If this doesn’t help, take a linear converter instead like LM7809. As it dissipates much more heat than its switching colleague, plan for mounting it to your metal enclosure.

Back to Project main page

CNC job setup

Schallbert — Mon, 22 Aug 2022

Demo video about CNC job preparation & execution

How-to: CNC job setup & quick demo↗

The following sections summarize preparation steps from the video.

Job prepare checklist

Securely mount your workpiece on the CNC
Transfer the job data to your CNC’s computer
Make sure Emergency Stop is engaged
Power up the CNC
Connect your computer to the machine and start the CNC software
Disengage the Emergency Stop
Prepare and check required tools for damages

CNC initialize checklist

Acknowledge the Emergency Stop release
Activate stepper drivers
Reference the machine
Change to the first tool you’ll be using
Set X, Y zero
Measure Z height / set Z zero
Put CNC back to home position
Perform spindle warmup run
Check Auxiliary devices (e.g. cooling, dust collection)

During the job

Supervise the machine
If needed, perform tool changes
If required, pause the job when something doesn’t seem right
Have the emergency stop in reach for worst case scenarios

Job completed

Set the machine back to home position
Engage emergency stop and disconnect the computer
Disable power supply to the machine
Take your workpiece from the machine bed
Cleanup

Measure Inrush current

Schallbert — Tue, 16 Aug 2022

Setting up the multimeter / oscilloscope / data logger

For this blog post, I was lucky to borrow a multimeter for energy measurement from GMC Instruments. It features a data logger so I can visualize the collected data.

Metrahit Energy

I’d rate the “Metrahit Engergy” to be more in the ‘pro’ than in the ‘hobbyist’ corner of multimeters, both its wealth of features and its price tag point in that direction. In contrast to lower-range models, it provides modes for instantaneous and average voltage/current, is able to measure power, power quality, and energy consumption of devices. It is also able to show total harmonic distorion of a signal and to read PT100 industry-grade temperature sensors.

This piece of equipment might come in handy for future articles as well.

MetraWin software setup

Download and install the MetraWin software to transfer and display collected data from the multimeter
Download and install Driver-Control software. It’s purpose is to provide a communications port between USB and the infrared communication interface adapter provided with the multimeter.
In MetraWin software, select METRAHit Starline Series (btw, unclear to me why the series’ name neither appears on the device nor the website)
Connect the infrared adapter. In the device manager (Windows-key –> type device and select device manager –> scroll down to Connections (COM & LPT), unfold), make sure the IR adapter shows up with a COM port ID). Select this ID in MetraWin’s Device --> Communication... --> SEL-COM window.

Multimeter setup for inrush current logging

In the Multimeter’s settings, make sure the address is set to any other value than 00 (Setup --> Set --> System --> Addr) because the infrared communication will not detect the device with an invalid value here (I experienced this and needed quite some time to figure out that not the IR adapter detection is faulty but the Multimeter’s settings). Background is that by address, the software supports up to 15 devices simultaneously.
Set the sample time to its lowest value for good impulse tracking: go to Measure --> Store --> Rate and select 0.0005sec which means the logger takes measurements with 2000Hz sample rate. This is as low as it gets, and these values already compromise the recording channel availability e.g. for energy measurement due to the device’s limited buffering ability.
Now select the value you want to measure (for me it is DC current in Ampère), hit MAN/AUTO and choose the range manually to a value higher than the peak value you’re expecting. Hint: If you just select A and the Multimeter shows TRMS or RMS somewhere, this is likely not what you want because with this option, the device will do some maths to get average values which naturally does not work when we want signal peaks.

For my amplifier’s inrush current, I had to set the range manually to >10A. This is because if the signal peak exceeds the ‘Auto’ range by far, accuracy will be low and the device might even record complete nonsense.

Alternative devices

Oscilloscopes would easily outperform the multimeter in terms of sample rate but as they have completely different design requirements, it’s fair to say that you cannot really compare these two device categories.

Nevermind, you can do the exact same measurement with the cheapest of scopes e.g. connected to a 0.1 Ohm shunt resistor as well, and set it to 1V/div. What for this specific task likely won’t work is a low-cost multimeter with “Data Max Hold” function. For capacitor charging on a DC power supply, the sample rate will be just too low.

Discussion of circuit to measure

The device under test is a 30W power amplifier board AA-AB32996 by sure electronics based on the Class-D integrated circuit TPA3110.

The supply path involves reverse voltage protection schottky diodes (three in parallel), connected to 6 bulk 220uF electrolytic capacitors on the positive supply rails of the amplifier IC. In addition, two different decoupling capacitors close to the IC reduce noise on that rail.

For comparison, the PCB layout example looks similar. You can see that the example features additional FB (ferrite beads) components for current leveling on the input rail and has less bulk capacitance for current surge handling capabilities. Also, the example works with more simple ferrite beads instead of “proper” inductor filters on the speaker output.

Motivation: Failed Prototyping with USB-C PD

For one of my prototypes, I connected the USB-C Power Delivery development board I discussed in this post and had to learn that it would blow up after just a few switching cycles - supposedly due to a high inrush current the power delivery device couldn’t handle.

That’s why I wanted to know how high the inrush current actually is and with which component I could effectively reduce the risk of voltage instability or even component damage when switching the amplifier on.

Actual measurement

I connected the multimeter’s current input to the negative power supply path of the amplifier, its GND connector to the power supply’s GND, +20V of the power supply to the amp’s positive power in, and the Multimeter’s voltage input to the IC’s power supply pins.

Then I selected SET --> Store --> Start --> Store to begin the measurement and connected the positive supply path. As voltage/current stabilized, I ended the measurement by pressing Stop.

Due to the high data rate, I had to transfer the data to my computer manually and couldn’t use the “live” function. To do this, I hit Read Memory --> Get Memory Info --> Move Memory Data to File --> save --> Open Memory Data File --> Display Data from File. Not entirely clear to me why the process has to be that complex - I’d like a shortcut to directly display data from the device - but nevermind, it works pretty well this way.

The Measurement Data File has a significant advantage over the Memory Data File: It contains metadata like scoped view, labels, units per division etc. When complete, a PDF can be generated with no big hassle.

Inrush current event

The whole event only took ~10ms, 20 samples. After a really high current spike of >12A, there’s a current plateau at around 4A until the capacitors are charged and the reduces to the circuit’s normal idle level.

The peak current has 1 sample only - I cannot say whether this is an outlier/ measurement error or a real event. Some assumptions for its cause:

very low ESR small bypass within circuits
Sample rate still too low
Current meter range violation
Power supply current limit engaged creating the current plateau (set to 5A so it does not really fit)

Voltage @Amplifier’s supply pins

The voltage measurement shows a steep incline for the first sample when the inrush event starts, followed by an almost linear increase until steady state voltage is reached after ~6ms.

These values correspond nicely to the current peak at the very beginning and the plateau showing the capacitors gradually charge until target voltage is reached.

Repetition & Conclusion

I repeated the measurement with different settings: Speakers connected/disconnected, amplifier in standby/idle but the results were identical. This led me to the conclusion that the amplifier IC and its periphery do not affect inrush current - it only depends on the power supply path and its protection, decoupling, and energy buffering function.

Drawbacks of doing this measurement with a multimeter are a limited data rate (I would have liked to measure with 10…100kHz for these surge events), the inability to live-view, and long data transfer times due to a (probably outdated) infrared communication standard. On the other hand, I am now able to do “offline logs” very quickly and without the need of a computer or even mains connection. Setup is easy and device portability is great for field application - I’ll do some more measurements in the future where the multimeter might be an even better fit.

For the amplifier, I will design a protective circuit as already discussed in this post to mitigate issues with USB-PD devices.

My CNC vibrates

Schallbert — Thu, 14 Jul 2022

Background

Lately, I have been pushing my machine closer to its limits. I’m now often formatting plywood full-slot with one pass meaning the machine has to take a Z+ of 12mm at F2700mm/min on a 6mm 2-flute rougher bit at S=24000rpm. I’m using climb milling for my projects in wood.

Now the machine sometimes starts vibrating.

You might say “just reduce machine load by setting a more shallow depth of cut”, “slow down your feed rate and spindle RPM”, or “Don’t take a rougher, that’s always inducing more vibration” - and I might follow of your advice at some point, but first I want to understand what’s exactly going wrong here and whether I can fix this issue without sacrificing machine time or cutter life.

Problem description

Machine vibration doesn’t only happen while formatting but also when I cut pockets, but it seems like only at certain positions during the cut. It feels like the whole machine is shaking just a little, but the sound of it makes me doubt that everything is alright. I captured two samples for your reference below.

Your browser does not support the video tag.

Vibrations manifest at certain positions of the machine while pocketing (80% clearing feed)

Your browser does not support the video tag.

Vibrations during formatting

Analysis

Cutting results are looking good to me still…

I retried with somewhat different settings: Reduced Z+ of only 6mm but increased feed of 3600mm/min with the same bit I used before - vibrations gone.

After contacting the machine manufacturer (refer to this thread in German), the most likely reason for the vibration is the router and its clamp that mounts to the Z-axis.

The router has less high-end bearings than heavy-duty spindles so they are subject to play. In addition, the 42mm standard clamp is attached to the Z-axis with two M8 fasteners while the router is clamped with only one fastener around its “neck”.

HF-spindles are most often flush-fastened directly to the Z-axis so the contact surface is incomparably higher, leaving little or no room for vibration.

Result

I have three options now:

Continue with high depth milling and tolerate vibrations and additional noise
Change to more shallow passes which solves vibration issues but adds machining time
Upgrade the spindle which adds considerable cost to my setup
Upgrade the machine bed to stiffen the machine frame

Image to Path

Schallbert — Mon, 04 Jul 2022

This is a short blog post about how to use the two Open-source tools Paint.NET and Inkscape to yield paths that can be used e.g. for engravings on a CNC machine, a laser cutter, or a plotter, even from a low-quality digital image in a lossy data compression format like JPEG.

It was inspired by this excellent Stackexchange thread.

Get the source image

Obtain the image you want to be working with on your machine.

Intellectual property

Note that images (even the ones you did draw yourself) are usually copyright protected so you must first get (written) approval from the owner of the intellectual property.

If the intellectual rights owner is you, that’s much more easy. This is why I chose an image I drew myself for this quick demo.

Image quality

Look at how ugly it is. It really shows I’m not an artist. Some lines are drawn multiple times and the contrast is low. Perfect conditions to show what the software can do!

But remember: Pretty much as always, the credo is “Garbage in, garbage out”. So you should make sure you get the best possible image quality to start with. Cleaning the image will be much easier this way and your engraving paths will both be simpler and more accurate.

Paint.NET

Groundworks

First, crop your image to the content you want to use later. Then take the luminance curve tool Adjustments -> Curves and try to separate highlights and dark tones as much as possible.

Second, if your image is as bad as mine, use the wand tool - play around with the tolerances - to knock out the background as well as possible.

Third, use the outline tool (Effects -> Stylize -> Outline) to get a nice and thick line from your image. Desaturating the image could help along with another contrast increase to make this line black. In case you start “losing” lines that are not dark enough - don’t worry, that’s easy to fix later on in Inkscape.

Cleanup

Remove as many “pixel clouds” and other artifacts from the image as possible using wand or eraser tools, it’s easier to do now than later.

Save your image as portable network graphic .PNG

Inkscape

Open the image in inkscape. Chose standard import settings DPI from file etc.

Tracing

Select Path -> Trace Bitmap. In the new pane, select Trace Bitmap, Single Scan Brightness Cutoff, and play around with the options. Make sure you select the image via mouse click before clicking update or apply. If you’re not happy with the preview, adjust the cutoff values. In case you’re getting lost given the multitude of settings, just have a look at this tutorial by Inkscape.

OK, now you have a lot of paths. Click Path -> Simplify to reduce the number of supporting points. Then continue manually removing artifacts that sneaked back in via the Brightness cutoff with the Edit Paths by Nodes tool.

Fill and Stroke

What we have now are two paths outlining the drawing on the outer and the inner, and a fill in between those paths that represents line strengh of the original drawing. As we want to keep the outer stroke / paths only, we’ll have to get rid of fill and inner line.

Select the Stroke and Fill tool in the lower part of the tool bar on the right. Select Stroke paint tab and set the stroke to Flat color. Under Stroke style, select a line width you’re comfortable with. Then go to Fill tab and set it to No paint. Now you’re only left with two outline paths of which you’re free to delete one if you wish.

Further cleanup

This looks pretty good already, doesn’t it? If you don’t mind having outlines more than a single stroke image, you’re done already. You could now further improve the image by pulling out paths or transforming curves. Or, if you want just a few lines to be cut, continue.

Use the break path at selected nodes tool before removing parallel lines so you don’t accidentally kill the lines you’d like to keep. Once you’re happy with the clean up and every wrinkle is ironed out, save your work as scalable vector graphic file .SVG.

That’s it! Now your CAM or any other machine toolchain you’re using can import the image (remember the DPI setting so your scaling remains valid).

CNC router overload

Schallbert — Sat, 18 Jun 2022

What I wanted to do

I had a request to cut and engrave high pressure laminates. These are a lasagna of compressed paper and phenolic resin, coated in a thin layer of (colored) melamine resin. HPL is really so durable that it’s even taken for facade planking. It is dishwasher safe and some variants can be heated to 200°C without taking any damage. You can find HPL as boards on playgrounds, garden tables, as panels in the tram etc. So I was like “cool stuff, let’s get started!”

Cutting data in High Pressure Laminates (HPL, “Resopal”)

I used the cutting data recommended by a couple of websites with values for “hard wood”. 2-flute upcut wood carbide endmill, speed of cut 450m/min, 6mm, Z+ 6,5mm, S24000rpm, F3500mm/min, full slot, climb milling.

Cutting results

And I was alarmed by the way the cut looked when I was done. Here are the symptoms:

While cutting, spindle RPM would drop repeatedly but catch up to desired speed again quickly.
high chipload on the bit, big chips, looking all fine
endmill didn’t follow a straight line: it somehow “wobbled”.
when cutting taps to keep the material in place, the cutter would move sideways a bit although it shouldn’t

Searching the culprit

First, I checked the spindle. No overheating, still tightly clamped to the portal milling machine, no bearing clearance, endmill securely fastened. Then I thought “Maybe something is wrong with the portal? Can I bend it?” No, I couldn’t. Then I looked at the endmill.

It didn’t look good at all. The cutting edges were practically gone where they had touched the material. Strange. It was brand-new when I started the job and it didn’t even run for 20 minutes.

Analysis results

I used much too aggressive cutting data
Because of that, the router motor was operated way outside its capabilities
This caused sudden drops of the motor’s rotational speed
This again led to increased chipload of my endmill
As bearings and attachment of the router to the portal are “soft”, the cutter tried to evade from its cutting path
This probably saved me from having a shattered endmill though

Improvements

Researching HPL parameters (again)

I did some more research on the internet. HPL is incredibly hard and tough. To compare some values (Brinell-hardness):

Beech: ~3.7N/mm² (Source)
HPL: ~185N/mm² (Source)
Mild steel ST70: 215N/mm² (Source)

I have found some recommendations for cutting HPL like “Carbide endmills should have a cutting speed of up to 150m/min” and “Use diamond-hardened cutters or 1-flute ones that are optimized for Aluminium”.

Interim containment

To correct things, I just did all cutouts again at finishing settings in CAM. With a fresh endmill and less aggressive feed rates of F2500mm/min and an updated depth per pass of Z+1,5mm. The result looks much better now.

Optimized cutting data

Still, let’s recalculate cutting data to take material differences to hard wood into account. I’ll choose a 4mm single flute (z=1) coated carbide cutter for aluminium and assume a cutting speed vc = 150m/min for HPL at a feedrate per tooth fTooth = 0,025 similar to cutting mild steel.

Target RPM:

$$n=\frac{v_c}{\pi d}=\frac{150000\frac{mm}{min}}{\pi \cdot 4mm}=12000\frac{1}{min}$$

Target feedrate:

$$F=n \cdot f_{Tooth} \cdot z = 12000\frac{1}{min} \cdot 0,025mm \cdot min \cdot 1= 300\frac{mm}{min}$$

I chose a 4mm endmill because my router motor torque diminishes over-proportionally when I reduce RPM. With larger endmills, this effect would probably lead to insufficient cutting power. If you have a spindle with an asynchronous machine, your torque curve is essentially flat so that shouldn’t be a problem of yours then.

The feedrate seems so slow - I hope that this wouldn’t heat up the material so much that it degrades (it won’t melt according to Wikipedia…)

For your reference: Here is the related thread in the forum that I am visiting regularly (all in German).

Further tests

I have made some parts with the bespoke endmill and settings. I optimized the settings a little for reduced cycle time and ended up with the following values: ftooth = 0.04mm and vc = 130m/min. For the 4mm endmill, this resulted in

S = 10300RPM
F = 410mm/min
Z+ = 3mm
vc = 1%

This endmill lived for about 1h. Not much still. Cutting with this kind of bit would never become economical. The HPL I am using has two material properties that will kill carbide cutters quickly. First, it is abrasive so the thin cutter edge is being beaten up. Second, the chips do not carry enough heat away from the cutter edge in time so that their sharpness deteriorates even more quickly.

Final solution

A more economic solution with 40 times (manufacturer statement) more lifetime might be to take a carbide bit with thick diamond blades (e.g. like this one). The downside is that it’s almost ten times as expensive. If I have to cut this material more often in the future, I might buy one and share my experience - in the meantime, I’ll rather have the cutters sharpened to save money.

CNC Spoilboard

Schallbert — Sun, 12 Jun 2022

Material list

Item	Count	~ Cost [€]
MDF, strength 19mm, 840x730	2	30
MDF, strength 5mm, 840x730	1	5
Drive-in nut	17	9
M6 Machine screw, 20mm + M6 hammer nut + washer	20	12
M6 Machine screw, 35mm + washer	17	5
Wood glue	-	0,10

Tools

Screw driver / wrench / nut runner
Your CNC

The Build

Download the drawings here
Adjust dimensions if needed ¹
Import it to your CAM tool. Provision for different material strength if you have.
Prepare sunk holes for your fasteners that attach to the machine frame to have some surface-planing reserve.
As both fence and spoilboard are larger than the CNC’s reachable area, you will have to use multiple jobs to complete the tasks. Choose your XY-zero points with care!
Also note that the slots will have to be cut “from the bottom” in case you don’t have a slot cutter.
Have your CNC do the work.
Glue together upper and lower MDF sheets so hammer nuts cannot fall through the slots

Back to Project main page

Note: the center stiffening bar does not have a defined position, so yours might be located somewhere else between its longer counterparts ↩︎

My Endmill 'screams'

Schallbert — Mon, 06 Jun 2022

Disclaimer

I’m working on my first-ever CNC milling project: A spoilboard that offers holes and t-nuts for workholding purposes and that supports an x-y fence and a tool length sensor.

I never used a spiral endmill before, but I bought a 6mm 2-cut upcut solid carbide bit with 21mm working length. And it just sounds awful. On my hand-router, I was exclusively using high-speed steel straight flutes and larger bit diameters. They sounded similar to the router itself, a loud but confident hum when cutting through material.

Beginner’s confusion

With this upcut bit, I followed the manufacturer’s recommendations of feeds and speeds for hard wood on my medium density fiberboard workpiece (depth per pass: 6mm, feed: 3500mm/min speed: 24000rpm). But obviously I do something wrong because there’s a terribly loud, high-pitched, uncomfortable screaming/hissing noise when I cut.

I gradually turned up the feed rate because I was under the impression that screaming bits “are not working at full capacity”. I ended up with 9100mm/min (!) where it was better, but still…

Your browser does not support the video tag.

Endmill screams while ploughing through MDF

Analysis

After the 15min job was finished I touched the endmill - Flutes are as good as new and the bit is barely warm to the touch. Hm. Also the router motor did not sound stressed. My steppers made the machine to literally plough through the MDF (smoothly and without any issue), I even had to turn the extraction up to not drown in dust. Also the result looks good to me; tolerances are tight and the cut is as clean as I’d expect it to be - much better than I could have done by hand.

Open questions

So what am I doing wrong? Is this sound “normal” for a small spiral flute bit? Does MDF need totally different parameters than hard wood for silent cutting, e.g. much less spindle RPM? Do I need to set a different ploughing strategy in my CAM tool?

Hold on tight, I’ll be checking with a forum to get some advice…

Questions answered

OK, I have an answer from the bit’s manufacturer (in German). This sound is normal for small endmills specialized in cutting wood. Well then. Probaby I should build a housing for my machine and buy some very good earplugs. 😬 Let me quote another tool manufacturer:

“Two-flute cutters are always noisier than single or three-flute cutters of the same size. Their 180° opposing cutting edges lead to additional vibrations of the cutter due to the alternately engaged cutting edges. In return, they offer better wear characteristics than single flute cutters and allow better chip evacuation than triple flute cutters.”

Comparison: Same material, different bit

Here’s a comparison video to judge just how different cutters can sound in the same material: 14mm 3-flute endmill face cutter, 10mm shaft, S5500, F7100, Z0.3, Fside 90%

Your browser does not support the video tag.

14mm face cutter calmly levels my spoilboard.

Just to test things, I used a spiral strategy in my CAM tool which takes 50% longer than the typical lawn-mower strategies, but this way there are no visible cutter paths on the finished material.

CNC Part5 - Macros

Schallbert — Wed, 11 May 2022

This is the 5th part of my portal milling machine sequel. It focuses on handy accessories that make your life as a CNC operator easier, and can lead to more quick and accurate milling results.

Overview

This article is about Macros for the CNC that help automate certain repeating operations. It will both cover basic and more sophisticated Macros for many purposes like automatic Z-zeroing, tool length measurement, or tool changes. It will touch some G-code commands and try to help you gain more confidence in your machine.

Disclaimer

The following subroutines have been written for RS274 NGC interpreter. I have validated them with EdingCNC as that is the CNC software I happen to be using. I have taken some inspiration from the macro file that both Sorotec and EdingCNC provide along with their machines.

As you might have different Hardware and Software distributors for your machine, these routines might not fit 100%. My intention was more to explain what’s happening in them than to provide a 1-by-1 copy paste template.

You might want to skip reading this article if your jobs are simple enough so you never felt the need to dive into CNC subroutines or in case your setup is 100% complete and you’ll never touch it again.

So what is a Macro?

A Macro or subroutine is a collection of instructions written for the CNC interpreter to perform actions. They are often used to automate recurring operations. On my machine, such a macro starts with a SUB command and ends with an ENDSUB. They likely include branching logic and G-code commands to have the machine do things or even show user dialogs to get the operator’s input.

On my machine, all macros are residing in a single file called macro.cnc that my CNC software will read on startup so it can be executed. Some of the software’s buttons even directly call macros from this file, e.g. for homing.

To demonstrate that this file matters a lot, close your CNC software and open the macro.cnc document with a text editor. Search for Sub user_9 (it should be a routine that doesn’t do relevant stuff), remove its contents and add the line MSG "hello world!" to it. Save your work, start the CNC software, and in the user menu, press button 9 - Voilà!

Starters: A simple macro to detect Tool Length Sensor status

Let’s get going and connect the newly-grown knowledge about macros to a useful example.

The tool length sensor

A tool length sensor is a device connected to an input of the CNC machine that triggers when touched (e.g. by a tool tip). There are many different designs available, ranging from a very simple microswitch with an enclosure and touch button to high-precision heavy steel sensors with automatic air purge operation to make sure the sensor’s surface is clean.

The tool length sensor is either mounted at the machine bed, on top of the spoilboard, or just placed at another fixed position on the machine. Mine is a very simple switch that I can put on any place of my machine. It has a claimed repeatability of 10µm.

Anyways, the actual magic is done in the CNC software.

Macro task

I want the macro to check whether the tool length sensor is operational (and not stuck). So I verify that the sensor switch’s status is “not triggered” - normally closed. If it was triggered, this could mean the sensor is stuck pushed (happened to me already) or has a broken sensor wire (didn’t happen so far), or just that the sensor is not connected to the machine at all.

Macro code

SUB is_sensor_ok
 IF [sensorStatus == triggered] ;# 5068 == 0 (normally closed)
 DLGMSG "Tool length sensor not connected or already triggered - please check"
 IF [dialogButton == 1]
 IF [sensorStatus == triggered]
 ERRMSG "Tool length sensor input still unexpected - aborting"
 ENDIF
 ELSE
 ERRMSG "User abort."
 ENDIF
 ENDIF
ENDSUB

The Macro’s name is is_sensor_ok. I’m checking sensor status and present a dialog message in case something is wrong here. Once the operator presses “OK”, I’m assuming that the problem has been taken care of and try again. If it is still triggered, I’ll abort the routine.

This Macro can be found in all sections below where I need the tool length sensor to measure something - It serves as a guard to not accidentally destroy my machine, the sensor, or the workpiece by interpreting wrong input.

Variable naming

The RS274/NGC language is old. I mean really old. Its first version was released in the late 1950’s. No wonder its parameters (#1 - #5399 is the allowed range) are all numeric both in naming and in the values they are able to store. No type system (like strings, int etc.) and no handy naming that makes understanding a parameter easy like sensorStatus.

It will just be #5068 and you’ll have to remember yourself that the CNC software manufacturer selected this variable to flag the tool length sensor’s status, and that it is boolean with 0 = not triggered and 1 = triggered.

To make the macro code as readable and comprehensible as possible, I refrain from using numerical parameter names in this article. A translation table can be found at the bottom for your convenience.

Using the tool length sensor to get workpiece surface (Z=0)

With this method, you don’t need to manually zero in workpiece surfaces by lowering Z-axis until the tool slightly scratches the surface and then setting workpiece coordinates.

Once the Macro is programmed and set up, just place the tool length sensor on top of your workpiece and jog your machine so it is placed directly above the sensor (not touching yet). Then start the macro. Your machine will now automatically lower its Z-axis slowly until the sensor switches. It will then reverse very slowly until the sensor untriggers. This point is then taken to determine Z-0 which is then set automatically.

Prerequisites

What we need before we can write the macro is the tool length sensor’s Z-position zTls at its switching point from being triggered to not triggered. I measured mine with a caliper and noted it down. We need this value so the machine can subtract it from the Z-height when touching off to yield the workpiece surface’s height.

Also, it is important to know how the switch is connected to the machine. The preferred way is “normally closed”, so the switch opens when triggered: triggered = 0. This is the more safe application because a broken wire or lost connection is detected automatically as the circuit breaks.

We’ll also have to determine touch probe forward feed and reverse feed, e.g. touch = 100mm/min, rev = 10mm/min.

The macro also features a more elaborate part: When Z-0 is about to be measured, but tool length is not known, the machine is able to store current position posX, posY in repositioning variables so that the get_tool_length macro can be called directly from here, and later repositionTo to where workpiece Z-0 is being measured.

The subroutine “Z-zero detection”

After switching the spindle off, the program will lower the machine’s Z-axis until it touches the tool length sensor (or until it has travelled down, spindle nose almost touchting the sensor where it would abort, claiming it didn’t find the sensor) via command G38.2. When triggered, it will reverse carefully until the sensor is released. This point is then saved as the new coordinate offset for the Z-axis via G92.

Code

before we start, make sure we’re in the correct state. Tool length should already be determined so future tool changes won’t require re-touching off Z-zero. Plus, we should know that the sensor is properly connected.

SUB measure_workpiece_z0
 IF ![toolLengthStat]
 DLGMSG "WARNING - Please first measure tool length!"
 IF [dialogButton == 1]
 xPosReposition = xPos
 yPosReposition = yPos
 repositionTo = 1
 GOSUB get_tool_length
 ENDIF
 ENDIF

 GOSUB is_sensor_ok ;# Check if tool length sensor status is OK

Let’s now program the actual routine by asking the operator whether Z-zeroing shall be performed now. When the CNC runs in simulator mode (without actual hardware connected), Z-zeroing won’t work so we also have to check this.

 DLGMSG "Start Z-Zeroing?"
 IF [dialogButton == 1] AND [operatingMode != simulator]
 M5 ;#Switch spindle off
 M9 ;#Switch coolant off
 G53 G38.2 Z[zSpindleTip + 5] F[touch] 
 ;# G38.2 = touch toward probe, stop on contact, flag error on fail
 ;# fail = 5mm before touching spindle tip
 IF [probeOk == 1] ;# #5067 == 1 : G38.2 command success
 G38.2 G91 Z20 F[rev] ;# now reverse slowly to find untrigger point (max. 20mm up)
 G90 ;# back to absolute coordinates
 IF [probeOk == 1] 
 G00 Z[zTouched] ;# #5063 Go to Z-axis's probe point	
 G92 Z[zTls] ;# Set Z-axis's coordinate offset (0) to tool length sensor's height
 G00 Z[zTls + 5] ;# clear sensor
 ELSE
 ERRMSG "Could not locate sensor untrigger point."
 ENDIF 
 ELSE
 DLGMSG "Could not locate sensor trigger point. Retry?" 
 IF [dialogButton == 1] 				
 GOSUB measure_workpiece_z0
 ELSE
 ERRMSG "User abort."
 ENDIF
 ENDIF
ENDSUB

Your browser does not support the video tag.

Executing the Get tool length macro on my machine

In this video, the machine is configured to return to XY zero when tool measurement has been completed. It would behave the same if I had the tool length measured after a tool change.

Measuring the tool length

If the machine knows the tool length before measuring the Z zero point, there is no need to re-determine the Z zero point during tool changes that occur whilst the job is in progress. I describe the relevant macro below.

Preparation

Most CNCs derive their current position data by counting the motor steps they have executed after completing the home run. To measure the tool length, a few programmed parameters are required:

Z-position of the spindle nose zSpindleTip (touches the tool length sensor without a tool inserted) so that the distance the tool protrudes from the spindle can be calculated
X-position xPosTls of the tool length sensor during tool changes
Y-position yPosTls of the tool length sensor during tool changes
Safety height zSafety (ensures that the spindle does not encounter an obstacle on its way to the tool change area)
Probing feed rate forwards and backwards. We can reuse this from the macro for the workpiece zero point.

The Measure tool length subroutine

Let us first define what this subroutine is intended to do:

Stop the spindle, coolant, etc.
Dialogue: The operator should enter the estimated tool length
Rapid traverse Z to a safe height, then XY to the position of the tool sensor
Probe the tool length sensor, similar to workpiece zero point determination
Calculate the tool length
Calculate the difference between the last and current tool lengths
Update the Z zero point accordingly
Return Z to the safety height in rapid traverse

Code: Measure tool

SUB get_tool_length
 GOSUB is_sensor_ok ;# Check if tool length sensor status is OK

 DLGMSG "Start tool length measurement? Please enter estimated tool length: " [toolLengthEst]
 IF [dialogButton == 1] AND [operatingMode != simulator]
 IF [toolLengthEst < 0]
 ERRMSG "Error: tool length cannot be negative."
 ENDIF

 IF [zSpindleTip+ toolLengthEst + 10] > [zSafety]
 ERRMSG "Error: tool too long - could collide with sensor."
 ENDIF

 M5 ;# Switch spindle off
 M9 ;# Switch coolant off
 G53 G00 Z[zSafety] ;# Go to safety height (machine coordinates)
 G53 G00 X[xPosTls] Y[xPosTls] ;# Go to tool length sensor
 G53 G00 Z[zSpindleTip + toolLengthEst + 10] ;# Move Z down to 10mm above estimated tool tip

 ;# measure tool length, save results, apply Z-offset if needed
 G53 G38.2 Z[zSpindleTip] F[touch] ;# probe sensor, latest stop point is spindle tip
 IF [probeOk == 1] ;# #5067 == 1 : G38.2 command success
 G38.2 G91 Z20 F[rev] ;# now reverse slowly to find untrigger point
 G90 ;# back to absolute coordinates
 IF [probeOk == 1] 
 toolLength = [zTouched - sTip]
 MSG "Tool length = " toolLength

 IF [toolLengthStat == 1] ;# defaults to 0 on startup
 lastToolLength = currToolLength ;# save last tool's length
 currToolLength = toolLength ;# save current tool's length
 toolLengthDiff = [currToolLength - lastToolLength]	
 G92 Z[zPos - toolLengthDiff] ;# Set Z-axis's coordinate offset (0)
 ELSE
 currToolLength = toolLength ;# save current tool's length
 ENDIF
 toolLengthStat = 1
 GOSUB reposition_spindle
 ELSE
 ERRMSG "Could not locate sensor untrigger point."
 ENDIF
 ELSE
 ERRMSG "Could not locate sensor trigger point."
 ENDIF
 ENDIF
ENDSUB

Your browser does not support the video tag.

Determining tool length on my machine

My machine is configured to return to workpiece coordinates XY0 when done with tool length measurement.

Tool change

When there is no macro for tool changes, the machine will pause its job and wait until you manually jogged it to where you perform the tool change, and requires that you re-zero the workiece’s Z0 position due to possibly changed tool length before continuing the job.

This task can easily be automated and the following section guides you how to write a Macro for this.

Prerequisites

An optional flag getToolLength could be configuring the tool change macro’s behavior - whether every tool’s length should be measured after a tool change or not. If you have an automatic tool changer, you might not need this to happen.

The following additional parameters are also needed

X-Position xToolChg where the tool change takes place (for me it’s xToolChg = xPosTls)
Y-Position yToolChg where the tool change takes place (for me it’s yToolChg = yPosTls)
re-use zSafety from get_tool_length macro
newToolNumber to indicate the requested tool from G-code (or for manual input)
currToolNumber to indicate the “old” tool to be replaced
toolChangeDone helper flag to indicate whether a tool change has taken place yet

The subroutine “Change Tool”

The following steps are being performed by the macro when a tool change is indicated either by command M06 “Tool change” within the G-code file of a job or triggered manually by the user:

Stop spindle, coolant etc.
If requested tool is the current tool, prompt dialog asking if it should anyways change.
Rapid move Z up (zSafety), then XY to the tool change position
Dialog: State current tool a and request to insert requested tool b
Check if tool change configuration implies tool length determination. If so, call get_tool_length macro.

Code

SUB change_tool
 toolChangeDone = 0
 M5 ;# Switch spindle off
 M9 ;# Switch coolant off

 IF [operatingMode != simulator]
 TCAGuard off ;# tool change area guard: off for tool change

 ;# handle case that tool is already in place
 IF [newToolNumber == currToolNumber]
 DLGMSG "Tool already mounted. Change anyways?"
 IF [dialogButton == 1]
 toolChangeDone = 0
 ELSE
 toolChangeDone = 1
 ENDIF
 ENDIF

 ;# go to tool change position and prompt to change tool
 IF [toolChangeDone == 0]
 G53 G00 Z[zSafety] ;# Go to safety height (machine coordinates)
 G53 G00 X[xToolChg] Y[yToolChg] ;# Go to tool change position
 DLGMSG "Please mount tool now. Old tool number: " [currToolNumber] 
 " New tool number: " [newToolNumber]
 IF [dialogButton == 1]
 IF [newToolNumber > 99] OR [newToolNumber < 0]
 TCAGuard on ;# tool change area guard: on for normal job
 ERRMSG "New tool number implausible."
 ENDIF
 toolChangeDone = 1
 ELSE
 ERRMSG "Tool change aborted."
 ENDIF
 ENDIF

 ;# prompt when complete and optionally call tool length measurement
 IF [toolChangeDone == 1] 
 MSG "Tool change from " [currToolNumber] " to " [newToolNumber] " complete."
 M6 T[newToolNumber]				 ;# set new tool number

 IF [getToolLength == 1] ;# config flag 0 = no, 1 = yes
 GOSUB get_tool_length ;# Measure tool length. Careful: To be called after M6 T !
 ELSE
 GOSUB reposition_spindle
 ENDIF
 ENDIF

 TCAGuard on ;# tool change area guard: on for normal job
 ENDIF
ENDSUB

Reposition (to saved coordinates)

This is a very short macro that can be called from other subroutines to reposition the machine either to a commanded position or to XY workpiece zero. It could be further enhanced with more repositionTo flag values, e.g. 0 = no repositioning, 1 = custom position, 2 = workpiece zero, 3 = machine zero …

Code

SUB reposition_spindle
 G53 G00 Z[zSafety] ;# Go to safety height (machine coordinates)
 IF [repositionTo == 1]
 G00 X[xPosReposition] Y[yPosReposition] ;# Move back to where requested before
 repositionTo = 0 ;# reset reposition flag and values
 xPosReposition = 0
 yPosReposition = 0
 ELSE
 G00 X0 Y0 ;# Move back to XY zero (workpiece coordinates)
 ENDIF
ENDSUB

Table of variables

In the following sections you can find all parameters I used in the macros above.

System parameters

Protected means that these parameters belong to fixed commands or states that are write-protected, and Reserved means parameters have a fix usage within my CNC software (don’t know about other CNC programs).

Variable name	Parameter nr.	Type	Comment
xPos	#5001	protected	current CNC position
yPos	#5002	protected	current CNC position
zPos	#5003	protected	current CNC position
currToolNumber	#5008	protected	[1…99]
newToolNumber	#5011	protected	[1…99]
zTouched	#5063	protected	Z where sensor touched
probeOk	#5067	protected	1 = OK
sensorStatus	#5068	protected	1 = triggered
operatingMode	#5397	reserved	1 = simulator
dialogButton	#5398	reserved	1 = OK

Config parameters

Config parameters are the one that are set once and then kept constant as they are tied to the CNC and their geometry.

Volatility of parameters heavily depends on the CNC software solution you’re using, the numbers of your Free parameters might differ. In my software, variables in the range #4000 - #4999 are persisted while all other free and reserved parameters are volatile and/or scoped.

Variable name	Parameter nr.	Type	Comment
triggered	#4400	free, persisted	1 = normally open
currToolLength	#4501	free, persisted	[mm]
lastToolLength	#4502	free, persisted	[mm]
zSafety	#4506	free, persisted	Z position for move
xPosTls	#4507	free, persisted	tls position
xPosTls	#4508	free, persisted	tls position
zSpindleTip	#4509	free, persisted	Z zpindle on tls
zTls	#4510	free, persisted	TLS height [mm]
toolLengthEst	#4511	free, persisted	[mm]
touch	#4512	free, persisted	touch feed [mm/min]
rev	#4513	free, persisted	reverse feed [mm/min]
getToolLength	#4520	free, persisted	Flag, 1 = Yes
xToolChg	#4521	free, persisted	tool change position
yToolChg	#4522	free, persisted	tool change position

Flag parameters

Free parameters can be used as wished by the programmer. Care has to be taken not to accidentally re-use an existing system variable, though.

Variable name	Parameter nr.	Type	Comment
toolLengthStat	#3501	free	1 = measured
toolLengthDiff	#3502	free	[mm]
toolChangeDone	#5015	free	1 = Yes
repositionTo	#5020	free	Flag, 1 = Yes
xPosReposition	#5021	free	reposition flag
yPosReposition	#5022	free	reposition flag
toolLength	#5024	free	[mm]

CNC Part 4.1 - Config

Schallbert — Tue, 08 Mar 2022

This is part 4.1 of my portal milling sequel. It is tightly coupled to Part4 but specifically handles settings for the CNC software that I use.

Hardware Assumptions

EdingCnc V5A-4E processor board or similar
Interface Advanced PRO breakout board or similar
Adapter Board or similar

These components are necessary to translate the commands created by the CNC software into physical-world signals that can actually interact with matter. The breakout board is controlled by the CNC processor and provides relay outputs for the router and my dust collector so I can have them switched on using my laptop. There are also analog voltage outputs e.g. to control spindle RPM and multiple inputs e.g. to read reference switch status and for signaling events back to the CNC software.

Of course there are many manufacturers of these circuits out there, but I happen to be using the above ecosystem so I can only speak for those. The underlying technology is the same, and although you might have a different user interface or communication channel like USB, your software will likely provide similar if not identical settings.

Setup screen 1

Kinematics

As discussed in-depth in this post, the green frame shows kinematics settings. A negative sign in the Steps/[mm] indicates that the stepper motor direction is inverted. Note that the reversing play field can remain 0 when ball screws are used. I’m using “simple kinematics” so the coordinate system is cartesian.

Reference / Emergency stop

This section controls how the software interptets signal levels for input switches on the interface board. Although I’m sure I wired all inputs “normally closed” to avoid undetected cable rupture, I had to invert the input for the emergency switch.

Spindle

These settings are very specific to the machine you’re using. I wanted the spindle to stop when I pause a job, and to automatically restart before continuing the job.

The approachFeedparameter is a bit confusing because the default value is really low here. Its meaning is the following: It’s the Z-axis’s downwards feed when a job had been paused and the Z-axis moved upwards. When you press “continue”, the Z-axis will be lowered with this feed and when at target depth, the job will continue normally. I entered 600mm/min as a value most endmills should be capable of dealing with.

Setup screen 2

User interface

I wanted referencing/homing to be mandatory before being able to load a job or to rapid-move the machine’s axis. I also selected automatic tool change although I don’t have a tool changer because otherwise my tool change macro helping me do the manual tool change wouldn’t be called.

Interface I/O

To account for the interface/ breakout board’s hardware setup, I had to invert some software outputs so they’d do what I’d expect them to. If I didn’t invert PWM1 for example, the spindle would start at full speed when I just wanted 5000rpm and vice versa.

Variables screen

Parking position G28 / G30

I set my parking positions in a way that allow me to work more efficiently. One of these commands would make the axes move to XY-zero of my fence that I’d also make workpiece-zero in CAM so zeroing in becomes really easy.

The other one is set to a point where the reference switches of all three axes are nearly pressed so homing the machine after startup becomes both easy and quick.

Variables monitoring

Note the monitoring window. You can enter a variable from the RS274 NGC interpreter range and have their current values displayed here. This comes in handy when you want to debug macros or just want to read internal parameters like measured tool length etc.

CNC Part4 - Setup

Schallbert — Thu, 03 Mar 2022

This is the 4th part of my portal milling machine sequel. It concentrates on hardware and software setup so the portal milling machine “Zerspanobert” can be soon switched into productive mode.

Kinematics

Before the CNC software can do any job on my BasicLine 0607, it needs to be set up correctly so it knows the mechanical ranges of the machines’ axes, where the homing switches are located, and in which direction to turn the stepper motors to yield “positive/negative” axis movement.

As a first step, a decision is needed on the machine’s axis naming (X, Y, Z), on where all axes shall effectively be 0, and movement direction (positive, negative). All those coordinate points are relative and can be chosen arbitrarily, but there are some conventions in the CNC world:

Axes:

X-Axis is on the portal bed, moving the portal’s Y-axis.
Y-Axis is up on the portal, moving the Z-axis assembly.
Z-Axis is located at the spindle, moving it up- and downwards.

Machine 0-point

Machine 0 and movement direction:

X-0 is at the “rear” of the axis. Movement direction is negative.
Y-0 is at the “left” of the axis. Movement direction is positive.
Z-0 is at the “top” of the axis. Movement direction is negative.

For my machine and the room setup I have, I chose the X-axis to be the portal’s long axis while the Y-axis would be along the linear guides of the machine bed. Z would be facing upwards, following the convention. I wanted the machine Zero to be in the “Top left corner”, so Z-axis would go down (-), Y-Axis down (-), X-Axis to the right (+).

Movement directions

To get started with the configuration, I connected to my CNC software. In the Setup view, I entered the step count needed to move an axis by one millimeter: 16 microsteps/step multiplied by 200 steps/turn divided by 10mm/screw turn = 320 microsteps/mm. I took double this value for the Z-Axis as the ball screw thread is less steep here.

16 microsteps/step is the stock value recommended by the machine manufacturer. There are some good arguments and an in-depth analysis here that recommend to use less microstepping, though. As I’ll neither require 1/100mm accuracy nor see extremely high cutting forces due to the materials I’ll be working with, I just kept the stock settings here.

Without homing (which wouldn’t work yet anyways), I used the jog keys at safety speed to check which key would move which axis, and into which direction. As my setup is different from “normal” the values I chose may differ from yours. If an axis wouldn’t match the arrow-key (e.g. pressing “right” and I’d expect my X-axis to move to the right, but actually the Z-axis would go down), I’d switch connections at the switch box until the keys would match the expected axes.

Now, if the keys would move the axis into the wrong way, e.g I press “page down” but Z-Axis would go up, I’d change the signedness of the resolution field. For the axes I now had:

X-axis -320.000
Y-axis 320.000
Z-axis -640.000

Machine travel

My machine has a usable mechanical travel of 0 / 732mm in X direction, -654mm / 0 in Y direction, and -134mm / 0 in Z direction. I entered these values into the mask for negative limit / positive limit and set the signedness based on the movement direction relative to machine zero.

Reference switches: Trigger level

The axes of most machines have a reference switch. These switches are used to “home” the machine. Homing is required at every machine startup because internally, the software is just inferring XYZ-position by accumulating motor steps. Once the system is restarted, all those values go back to 0 so the machine forgets its position.

I connected my switches so they are “Normally Closed”: Their contacts are conducting current when not triggered. This way, a faulty open connection can be detected immediately as the machine would home (without success) at an unusually low speed in this case: homeVelocitySlow is just 1/10 of normal homing speed in my config. Still, we need to make sure that the homing contact is interpreted correctly. This can be set in the “Homing and E-Stop” section of the “Setup” screen. I would enter a 1 for normally closed.

Reference switches: Setup

Some CNCs provide a closed-loop positioning control - they don’t need reference switches because their stepper motors are equipped with position encoders that feedback the absolute position of each axis. These machines will not lose steps and are able to detect positioning errors, e.g. caused during mechanical overload.

Using the reference switch, a machine without closed-loop positioning can reset its step count at a known position so that axes movement becomes repeatable. Still, the reference switch position has to be known to correctly relate step count to axes positions, and that’s why we need to enter the reference switches’ positions on the axes. I just watched by how far I could move the machine (Jog menu) after the reference switch has been triggered before I hit an axis’ mechanical end. These are the values I finally entered for my model:

Steps/AppUnit

X-axis: 10mm
Y-axis: -4mm
Z-axis: -16mm

Emergency Stop

The emergency stop button is just another input for both the CNC software and hardware. Pressing this button will stop axis motion and deactivate both spindle, cooling, and auxiliary circuits. It is also wired “normally closed” to detect connection issues and can be configured like a reference switch.

When the emergency stop has been activated, the machine’s drivers have to be reset and the machine must be homed again before it can continue normal operation after the button has been released again.

Axis motion parameters

The CNC software needs to know how to correctly handle motion of the machine’s axes: a heavier and stiffer machine with strong steppers naturally will be able to run more quickly and handle hard accelerations / jerks better than softer, lighter machines under high axial load conditions during cutting. This is why motion parameters of your machine should be carefully identified and deposited in the software.

Axis speed

“Why should Zerspanobert be fast?” I asked myself. I didn’t come up with too many good answers as normal feed rates with the endmills I plan to be using will remain far below the machine’s capabilities. Nevermind, here’s some arguments to make a machine quick:

job times should not become too long
high travel distance to be covered on a large machine
when cutting soft, low-density materials, feeds may be much higher

Still want a quick machine? Some calculations will help us here.

Determining motion electronics capabilities

The maximum speed is limited by multiple factors:

Parameter	Symbol	Value	Unit
max drive frequency	fD	125	kHz
drive microstep setting	N	16
max software frequency	fS	125	kHz
stepper steps/turn	nS	200
stepper max rpm	nSm	3300	1/min
ball screw thread width	P	0.01	m
ball screw max rpm	nLm	1700	1/min

Let’s take an example for a 125kHz driver and assume that the software supports that step frequency. I further assume the motors have 200 steps/turn and your driver is configured to 16 microsteps/step.

This gives you a maximum speed of $\frac{125000}{3200} = 39$ rotations per second or 2340 RPM which is quick for a stepper motor. On a 10mm thread, this would lead to a machine max speed of 23.4m/min which is really, really fast (for non-professional grade machines this guide is targeting).

Checking mechanical limitations

Ok, let’s see if that’s acceptable for our ball screws - their allowable RPM is determined by series (KGS16 in my case) and unsupported length (~850mm). The value I read from a manufacturer’s online documentation is around 1700rpm.

So, 17m/min remain. This is still fairly fast - I’ll be far away from full motor torque availability as motor torque reduces over speed. Let’s have a look at the stepper motor’s datasheet to make sure we’re safe.

Verify motor torque is sufficient

Datasheet: My stepper's torque curve (Image courtesy of Sorotec GmbH) ^[1]

The blue arrow indicates my most-often-to-be-used feed rate for roughing in wood where there’s plenty of torque available for large depth increments. According to the green arrow, I still have nearly 50% torque available during quick smoothing or while working with soft materials like foams. I decided to stay 30% below critical speed of the machine (limited by the ball screw) for my G0 commands that won’t run under spindle load. That’s where the yellow arrow is located: Maximum velocity is 12m/min = 200mm/sec. Here, only as much as 0.7Nm are available which is 1/6 of original torque.

This way, I have good confidence to enter velocity data for the axes X and Y. For Z with half the thread height, I divided it by 2.

Axis acceleration

Imagine you want to cut a 90° corner in your program with a feed rate of 12m/min. Not unlike you when riding a bike and taking a sharp turn, the machine is not able to brake to zero on one axis in no time and accelerate on the other one simultaneously. If the axes don’t wait for each other, we’ll cut the corner round. If we do wait, the feed rate is not constant anymore - a dilemma. That’s why we need to find appropriate acceleration values.

For your bike, decelerating “appropriately” might mean to not risk wheel lock or bicycle fork fracture but on the other hand still quickly enough to not “miss” the corner. Same thing for the CNC: We don’t want it to lose steps, resonate, block, jump or twist too much while maintaining a profound operating speed, high accuracy, and low machine wear.

Sounds impossible? That’s why we will spend some more time here.

Your browser does not support the video tag.

Cornering: `vmax = 200mm/s`, `a=400mm/s^2`

Take the above example: I used a pen to visualize acceleration behavior of my CNC with some conservative acceleration values. For the last path, The CNC slows down to 0 so it can take the sharp corner. All the paths before it also won’t meet the targeted 200mm/sec but the larger the radius is, the quicker the machine ran.

Motivation

“Why should Zerspanobert be accelerating quickly?” Here I have more answers. I’ll be doing a lot of short-line and 3D work. They often involve many lines of code and thus the CNC will have to change direction and feed rate all the time, maybe even use Look-Ahead Feed or G64 commands.

I do also want to keep running through material with an almost constant feed rate to not generate “hot spots” where the machine dwells before accelerating into another direction. That’s why, with at low acceleration settings, these tasks will either take long or be less accurate than I’d like them to be as look-ahead will slur trajectories.

On the other hand, high accelerations will destabilize the machine and might lead to jerky behavior - but we’ll cover that later.

Determine maximum acceleration formula

As we did for velocity calculation already, let’s juggle some formulas to get a do-not-exceed acceleration value. We need

Newton’s second law (axial movement): $F_{ax} = m_pa$
Inertia torque of the axis: $M_a = \frac{a J}{P}$
Moment of inertia of the axis (cylinder): $J = m \frac{a}{2}r^2$
Motor torque required for linear movement: $M_d =\frac{F_{ax} P}{2 \pi \eta} + M_a$

To calculate the maximum acceleration, I combined these formulas to

$$a = \frac{M_d}{\frac{m_p P}{2 \pi \eta} + \frac{J}{P}}$$

In words: The acceleration the machine can theoretically reach is the available motor torque $M_d$ divided by the portal’s mass $m_p$, weighed with ball screw specific parameters like thread width $P$ and the overall motion system’s efficiency $\eta$ plus the moment of inertia $J$ the rotating axis components have, weighed again by thread width.

Working with values

Let’s add some data - for my BasicLine 0607:

Parameter	Symbol	Value	Unit
stepper torque @ vmax	Md	0.5	Nm
portal mass	mp	40	kg
ball screw thread width	P	0.01	m
ball screw radius	r	0.008	m
axis’ rotating mass	ma	4	kg
axis system efficiency	n	75	%

Where I didn’t know the exact values, I took worst-case data. For the stepper motor, assuming maximum velocity and lower voltage is worst-case because its torque will be much lower than at standstill at no load. I just guessed my portal weight and the rotating mass of my longest axis plus stepper’s core. Typical ball screw efficiency is at 90% but I had to take both bearings and losses of the pre-stressed linear guides into account. I conservatively estimated the axis system efficiency.

Ok, here’s what those values return in the formula as a maximum acceleration for my BasicLine CNC:

$$a = 5.0m/s^2$$

What this value means

Let’s see what time the machine would need to accelerate from 0 to top speed. Now good old law of uniformly accelerated motion comes in:

$$v(t) = at \Rightarrow t = \frac{v_{max}}{a} = 40ms$$

How much distance do we cover to accelerate from 0 to full speed ($s_0 = 0, v_0 = 0$)?

$$s(t) = \frac{a}{2}t^2 + v_0t + s0 \Rightarrow s(t=40ms) = 4mm$$

What axial forces does the portal have to take during acceleration? $F_{ax} = m_pa = 200N$

The acceleration I calculated here is more than a magnitude larger than the manufacturer’s default setting 300mm/s^2. I wouldn’t have expected such a big difference.

🤔 I must have overlooked something.

Conclusion: A matter of experience

The formulas neither take machine stiffness into account nor do they show how the machine would twist or jerk under dynamic load. What is Jerk, you ask? View this short youtube video for an explanation in CNC context. I didn’t find any point in my CNC software where I could set values for allowable Jerk, though.

Although I learned a lot about how math can help guide us here, it all boils down to experience again:

How heavy and sturdy is the machine?
What kind of material am I working with: Is it forgiving or stiff?
Do I have complex, possibly 3D jobs to do?
Does my CNC live in a production environment where machine time is important?
How does the CNC sound and feel with high acceleration values?

I entered a value of 800mm/s^2 as acceleration for the X- and Y-axes. It seemed to work without issues whatsoever. After all, it remains below the calculated motion system’s capability by factor 5.

Tryout

I know, this was a long read and much more an educated guess than conducting strict math 😅. Still, we’re now ready to home the machine the first time. Try it. After that, use the jog menu to go to Machine 0 on all axis. Does it stop before hitting the mechanical end?

Then use the continuous jog to move close to the other end of all axes. Carefully jog the last millimeters. Are measurements and settings all correct? If so, try the 100% speed modifier key (Shift - on my keyboard). Does the CNC move rapidly and automatically stop at axis end without collision?

Congratulations!

Max motion settings calculator

You bravely read the whole article. Kudos! Be rewarded with a little Excel tool: CNC kinematics helper you can download and use to work through some numbers for your machine. Needless to say that I cannot take any responsibility for the results you get when using that tool or any harm that wrong values can do to your machine or even to your health.

You want more?

Machine software setup

CNC Part3 - Build

Schallbert — Tue, 01 Mar 2022

"Zerspanobert" CNC assembly↗

This is the 3rd part of my portal milling machine sequel. It concentrates on building the machine from an assembly kit.

Brand: Sorotec, Product: Basicline 0607

My portal milling machine arrived as an all-inclusive assembly kit in a huge package on a small pallet. After inspecting its contents, I was pleased to find out that all major parts were packed so that I could start the build immediately.

Machine facts

Fully built, size is roughly 1m x 1m x 0.8m
Work area: ~ 630mm x 730mm x 135mm
Weight without spindle, mechatronics, and accessories: 52kg (source: manufacturer)
ball screws and linear guides on all axes
Steppers: 3Nm / 4.2A @48V
Speed & Acceleration config: 200mm/s, 400mm/s²

Assembly

The portal milling machine’s mechatronics assembly took ~16h including stepper motor wiring, terminals, connectors, but excluding the switch box I wrote about here which I spent another 8h on.

The following sections each describe a construction stage, and also mention the things I struggled with.

1. Preparing the ball screw bearings

A ball screw (youtube link) is a mechanical actor that converts rotary into linear motion. Instead of direct contact to a nut (like a normal screw), the nut houses a system of revolving balls that reduce friction between nut and screw to a minimum. On the CNC, they are used to move the portal with help of a stepper motor.

The ball screws need two bearings, a fixed (stepper motor side - black block on the images), and a loose one on the opposite side that just holds the screw in place. Mounting the loose-side ball bearing into its fitting proved to be more difficult to me than I’d have imagined. Im not experienced with metalworks at all, so I had a hard time hammering the bearing in as the fitting was so tight. I accidentally shaved off a bit of the aluminium fitting and the result was terrible.

Your browser does not support the video tag.

Fitting of ball bearing gone wrong

Luckily I had a 20mm ratchet nut that fits the outer ring of the ball bearing. After heating up the fitting a bit, I was able to use the nut, carefully hammering the bearing into the fitting with better results.

Your browser does not support the video tag.

Fitting of ball bearing after reinstall

Let’s hope the bearing didn’t get polluted too much with aluminium chips.

2. Machine frame

The frame is made of “standard” aluminium profiles connected with brackets. Additional stiffening is provided by massive aluminium plates on front and rear of the frame. Linear guides are mounted to the sides of the frame which later keep the portal in place, allowing movement in X-direction of travel.

It took me quite a long time to align the frame properly so that every angle would measure 90° and I’d still have some “air” at the right hand side for later arrangement as the gantry would dictate the exact frame width. Problem was that when tightening the screws from one side to the other, somehow the frame would always go out of square and I would have to loosen the screws a bit, then tighten up again, only to see that nothing much had improved.

After a couple of hours of cursing under my breath, I tried to tighten all screws just to 3Nm first, then to 6Nm, 9, etc. This worked very well so finally I reached the targeted 25Nm and was still happy with all the square angles 😅

Mounting front and rear plates, the ball screw for the Y axis, and the reference switch then proved to be quick and simple in comparison. The image above shows the fully mounted Y-axis. Linear bearings that will later carry the portal are already mounted on linear guides. Here’s how the trollies look like on the inside:

Your browser does not support the video tag.

Linear bearing with revolving ball system

3. Gantry

For this machine’s design, the portal moves in X-direction and provides a flange to the Z-axis that itself can be moved on the X-axis via a stepper motor on the portal’s left hand side.

First, the portal cheeks are mounted on the linear bearings of the Y-axis. Locator pins that the manufacurer designed were a great help to make sure that they are positioned correctly. Then, linear guides for the X-axis are attached to the crossbeam of the gantry which is then finally hung into the portal cheeks and secured with cast brackets. To keep the crossbar as close as possible to the cheeks before fastening, I used screw clamps.

This machine works with a single stepper motor for the Y-axis, not with two independent ones on each portal side that can be seen on other designs. To make the Y-axis’s ball screw move the portal, a girder is mounted underneath the portal cheeks that connects to the ball screw’s nut. It’s easy to spot in the video below. As the assembly was a bit heavy already and I had to access the underside of the frame, I used a wooden stand to carry one side of the frame while working underneath. This way I could avoid turning the whole machine upside-down.

Your browser does not support the video tag.

Easy portal moving by hand - without ball screw attached

After loosening some of the frame’s fasteners to readjust the air-gap with mounted portal, I made sure that the portal could move without difficulty. Then I tightened all relevant fasteners for portal and frame again, using the 3Nm-increment-technique I introduced before to avoid losing square.

Next step was to mount X-axis ball screws, linear bearings, and flange plate to the portal which luckily was easy enough to do.

Finally, the X-axis reference switch could be added along with the motor socket.

4. Z-Axis

As the Z-axis is a component of its own, I assembled it aside the rest of the machine. Locators would simplify finding correct positions for the bearings, and a single linear guide would keep the Z-axis in place.

I’m pretty sure that this is fine for woodworks, but I don’t know if the stiffness provided by this Z-axis design is enough to routinely work with more challenging materials like Aluminium - for that, maybe a heavier alternative with two linear guides on the Z-axis should be used. Nevermind, I wanted a machine for woodworks and plastics - you get what you pay for.

It’s always nice to see when things really fall into place 🤗 - Look at how snugly the individual parts of this assembly kit fit together!

Your browser does not support the video tag.

I like the sound of nicely matching parts!

When component assembly was complete, I could mount the Z-axis to the gantry. Then I aligned the Z-axis with help of a dial gauge. What I observed: Even if I had invested much more time in further improving positioning errors, it is enough to push the Z-axis at the bottom end with my hand to increase deviations to above 0.01mm easily. Unless you have a very heavy (annealed steal) machine, don’t expect wonders.

Assembling the stiffening cheeks to the Z-axis (image below) was more tricky than I anticipated. The hammer nuts were so small and the fasteners so short that there was not much room left for wiggling things in place - I had to align again and again to properly position them for sliding the reinforcement cheeks onto the assembly. When this was complete, it was an easy task to mount the Z-axis reference switch.

5. Mechatronics

It’s time to mount the stepper motors I prepared earlier . I lubricated the claw couplings between motor and ball screw with petrolatum and fully pushed it on the screw’s shaft before adding the motor. On the X-axis, fastening the setscrew was fiddly because the hex key has very little room to act on the motor side. If I had to build this machine another time, I’d do it like this to eliminate the issue:

Take the clutch apart
Fasten one clutch half to the ball screw
Add the other clutch half to the motor shaft, don’t tighten yet
Put the motor into its mount, only tighten so much you can still move it by hand
Push the motor’s clutch half into the screw’s
Take the motor off and fully tighten its clutch
Now mount the motor

As the motor wires are not very long, there is a small junction box located closely to each motor where both reference switch and motor power wires are connected to drag chain compatible control lines. Every stepper motor also gets a dedicated earth wire leading to the switchbox.

Once this was done for all three axes, I mounted the drag chains. Here, it’s a good idea to thoroughly select the mounting direction because the chains can be opened for maintenance in one direction only. Example: if you need to add or exchange cables later on, it’s frustrating to see that there’s no way of getting the screwdriver in place to unlock the chain links.

Also, please leave the control lines longer than needed and shorten them as a last step before soldering the connectors. For example, the required length of Z-axis wiring may come close to 5m when the switchbox cannot be placed immediately next to the machine.

6. Lubrication

Finally, I followed the (well-made) maintenance manual to get the machine properly lubricated.

Did I mention that I’m not a metalworks guy? Working with the grease gun proved to be pretty frustrating to me. At first, I didn’t manage to get anything out at all and finally, it wouldn’t stop. Now I’ve got a very-well greased working table but a dry machine.

And when is it enough grease? Are two shots into the grease nipple sufficient? I cannot even check whether anything actually entered the linear / ball screw bearings (they have a non-return valve), but the nipples are completely slurred from the outside.

Well, let’s assume everything is fine 😜

7. Missing / bonus parts

As written above, all vital parts of the machine were packed Ok. But some small parts were missing for my machine configuration:

18x 18mm M5 hex fasteners (linear guides X-axis)
6x 0.75mm cable shoes to get the earth wires mounted
4x 22mm M4 hex fasteners (stepper mount)

But I also had some bonus parts

1m drag chain extra
4 M5 washers
9 M4x10 countersunk hex fasteners

I’d recommend to add 100 zipties to the assembly kit so all wires can be kept nice and neat. Cable sleeves would be better still but likely come at higher cost.

8. Critics

I like the construction manual (Version 2.1.5) very much. It contains a parts overview and each step is explained in high detail. Also the build process is well thought-through and I as a non-professional mechanic didn’t have bigger troubles to correctly assemble the machine.

The machined parts have a high quality and are free of ridges or sharp corners. All necessary threads had been cut in advance - even the ones for the optional mechatronics kit - so that I didn’t have to buy tools or gain knowledge about how and where to mount them.

What I didn’t like is that some fasteners were missing holding up the completion for a weekend, and that the expected fastener count wasn’t printed in the manual so I only realized there are parts missing when it was already too late. I understand that this machine can be ordered in different sizes and configurations so it might be too much to expect a construction manual for each and every combination.

Some of the bigger aluminium parts arrived with scratches as the pallet somehow must have broken and torn through the packaging during transport. To me it shows that either the logistics company or the manufacturer didn’t pack the machine 100% safe or maybe underestimated its weight.

A last point about these grease nipples (still no metalworks guy): The one for the Y-axis cannot be turned anymore once the ball screw’s nut has been mounted to the lower gantry beam due to missing clearance. This is unfortunate when lubrication suddenly has to be done at a different angle than anticipated. Also, I’m pretty sure the lower grease nipple of the Z-axis points into the wrong direction in the manual.

I tried my best to find things to gripe about but came up with only so much as nitpicking. None of my critics limit the functionality of the machine and neither cost a considerable amount of time or money to fix. So far it looks like I bought a sturdy device that will help me do my prototyping woodworks more precisely and much more quickly than ever before.

This is Zerspanobert!

9. Let’s put the machine into operation!

You want more? Machine setup for first use

CNC Part2 - Electronics

Schallbert — Fri, 18 Feb 2022

Portal machine electronics

This is the second part of my Portal Milling Machine sequel. It is all about the electronics needed to control a CNC machine’s movement.

AC / Mains components

First, it houses main fuses for the whole machine control and a central power switch. It may contain mains-connected relays to control loads like spindles, cooling, dust collection. For these circuits, sub-fused power paths and outlet sockets might be available.

Second, it has a power supply unit for the low voltage DC rail. Its voltage and power output will depend on your stepper motor load, control/breakout board voltage requirements, auxiliary low-voltage circuits it has to drive, and the region you’re living in because standards and regulations tend to differ.

Third, it may have additional signal relays to pair a variable frequency drive or similar.

Variable Frequency Drive

A VFD is commonly known as a device that controls a 3-phase alternating current machine’s speed of rotation. Supplied with 1-phase or 3-phase AC, it buffers energy in a DC intermediate circuit from where it is then forwarded to a 3-phase H-Bridge that creates a 3-phase AC output with variable voltage and frequency.

As this 3-phase AC voltage is generated by transistors in switching mode which enormously decreases transformation losses in comparison to linear operation. The output voltage is quantified in time and magnitude though (“digital”), causing undesired circuit feedback and line noise both on the in- and output of the VFD.

Almost all professional CNC spindles are 3-phase AC asynchronous machines connected to a VFD.

VFDs often have a standardized 0…10V analog input to control machine speed, enable inputs that are fed e.g. by a signal relay of the numerical control’s interface board, and an emergency halt or error signal output to the CNC interface.

As of now (2022-Feb), I don’t own a VFD. So I don’t know any details about tuning, parameter setting, and customization for a certain spindle to write about.

ELV / DC components

The typical “small” CNC’s stepper motors are driven in the PELV range (Protected Extra Low Voltage) with direct current (DC) and below 60V so no additional touch protection is needed. The “P” in PELV stands for “protected” which means that the devices are connected to grounding wires. This way, a short to ground will safely turn the circuits off, blowing the fuses.

The stepper motor drivers, one each per axis, are also placed within the box. They are typically SELV (Safety Extra Low Voltage) so that they don’t need an extra grounding.

Then there’s a signal board, breakout board, or relay board that forwards stepper signal trains to the drivers, controls auxiliary relays, reads reference switch and emergency stop inputs, and output signals for spindle, cooling, dust collection, and/or auxiliary controls.

A fan keeps temperatures within the switch box low. Maybe there also is an interface board mounted in the switch box that is controlled directly via the CNC software, e.g. through USB or ethernet. It translates and routes information to the CNC’s individual components and back to the program, e.g. for reference switches or emergency stop.

Stepper motor drivers in detail

The stepper driver interface is quite simple. It takes TTL level of 0V (low) and 5V (high) and is defined by only three inputs:

Signal name	Description
Enable (EN)	Enables power to the drives. E.g. allows active position hold
Direction (DIR)	Select direction of motor rotation
Pulse (PUL)	Makes the motor turn a (micro)step

Interestingly, the same three input signals are needed to control a floppy disk drive’s reading head. I did this in one of my earlier projects, a floppy-bass-organ based on the software project moppy.

Internally, the driver interprets input signals and outputs higher voltage, high current pulses that are suited to drive the stepper motor. Two H-bridge channels - circuits with four switching transistors to control direction and magnitude of current flow through the motor’s coils - called A and B provide energy to the pole pairs of the stepper motor.

You can find more detailed information on stepper motor drives, microstepping, and other aspects e.g. here or on this blog (in German language).

Stepper motors

Stepper motors - like 3-phase AC or brushless DC motors - are electronically commutated. So direction and magnitude of current pulses have to be provided to the motor to make it turn continually. Most stepper motors nowadays are hybrid motors that combine a multi-toothed rotor, split into two halves that are shifted to each other by half a tooth, each controlled by one coil path. This kind of motor has four wires at least, two for coil A and two for coil B. Unifilar hybrid motors even have eight leads that again split the coils into two parts - this way, they can be connected in series, in parallel, and also unipolar so each of the four coils can be controlled individually.

The most common way to connect a stepper motor to a CNC stepper driver is the “bipolar parallel circuit”: Here, the two coils of a pole pair are connected in parallel with the same orientation. The two wires from this parallel circuit are then connected to the driver’s A+ and A- outputs. This wiring is repeated for channel B. I twisted the wire pairs that belong together with a power drill because I was too lazy to do this manually.

Unlike 3-phase AC or brushless DC machines, priority for stepper motor design does not lie on highest possible energy efficiency, low noise emissions, high rotation speed or thermal management for continuous operation, but on torque for low speeds, high clamping forces to securely hold a position, and smooth operation at low speeds. Stepper motors are built for high acceleration and shall always follow the speed of pulses as the machine’s positioning depends on that.

Unlike servo motors, where a closed-loop control guarantees exact positioning by measuring the shaft’s position, the stepper motor’s position is assumed. The steps are counted and multiplied with the known angle the motor turns per step. This works very well as long as the motor’s maximum load is not exceeded in which case steps are lost, creating position errors.

The build

Your browser does not support the video tag.

Switchbox build in 9 seconds

The build of the switch-box and wiring assembly kit that I bought took just about 8h of my time. It proved crucial to thoroughly read and understand all of the circuit plans and step-by-step instructions provided in advance to avoid mistakes that could otherwise cost hours to fix.

About the kit I like that the components are electrically matched and all necessary wires, sockets (apart from an ethernet socket), and the documentations were complete. I only missed cable straps and some lugs to simplify cable management and connections to the housing. The housing is very robust and all sockets fit in their places nicely.

Most painful when building where the solder joints of the stepper motor/D-sub connectors and their insulation. It was also not very easy to mount the numerical controller board in the switchbox.

I was really excited when I first turned the switchbox on. Was I relieved when the fan started working and all sorts of LEDs lit up to indicate a working system.

Your browser does not support the video tag.

Switchbox dry run: Checking emergency off

Safety notice

No, an opened mains powered device shouldn’t be turned on unless you know what you’re doing. Yes, I have a residual current device installed in my tech shack to protect myself from hazardous voltages, and yes, I had all fuses double-checked, did quite some visual inspection and performed continuity measurements on every single connection to exclude miswirings. Luckily, the control box passed the initial function test. Both the emergency signal state was correctly interpreted and also the CNC board’s LED flashed with the expected 1Hz rate.

Building a “Mini Control / Series C1”

For the Maker Faire Ruhr 2024, where I was one of the exhibitors, I built another machine control box: The simpler “Mini Control / Series C1”. I have embedded a time-lapse video of it here:

CNC electronics assembly↗

Advantages:

Quick assembly
No soldering
Many snap-in parts
Small housing

Disadvantages:

Connection to the machine is via screw terminals; Separation from the machine not possible
Loud fan, active even when idling (but can be solved with “Schallbert’s Fanhack”)
Several external power supplies (stepper motor supply, 24V supply, spindle switch box)

I did a lot with the associated small CNC milling machine: Tuning for shorter production times, a specialization for engravings and various projects such as stamps, dithered images and multi-colored carves.

CNC Part1 - Portal milling machine

Schallbert — Sat, 12 Feb 2022

CNC Portal Milling Machines - a beginner’s view

My first computerized milling machine will be delivered very soon, in form of a lot of packages with even more parts inside that require assembling. I ordered it to speed up my AnywhereAmps prototyping process. And of course to realize some other hardware projects that I had in mind, but was never able to build. Yet.

Wait, what?

Oh, Ok. CNC means computer numerical control and describes how the tool is controlled - not by hand but by code.

This is called G-Code, a widely used language to tell CNC machines like my portal milling machine what to do.

A portal milling machine is like a router, just not hand-held. It is mounted on an apparature that can move the router in all three axes X, Y, Z with the help of stepper motors or servo motors so that a computer can control its movement and cut any path you like. Its way of operating is similar to a 3D printer, although the mechanics of the two have completely different requirements - a portal milling machine has to be constructed much more rugged, heavy, and stiff than a 3D-printer as it has to take all forces created by cutting through material.

Of course, CNC machines are not limited to 3D printers and routers. There are CNC lathes, plotters, laser cutters, water jet cutters and many more. Strictly speaking even 6+ axis industry robots are CNC machines.

Questions

These are some of the questions I had before actually buying this pretty expensive piece of technology. Some of them are easy to answer in just a few lines, but some likely take a whole article on their own.

CNC router decision

Does it really speed up my prototyping process?

I don’t know yet, but I hope so. Consider this: Woodworks for a single prototype take roughly 8h of my time. Most time consuming are preparations for cutting, measurements, and tracing. What takes more than an hour of manufacturing time alone is AnywhereAmps Alpha’s speaker adapter with its curved surface.

With a CNC portal milling machine, I’m hoping to become more like a biscuit maker: Nesting all parts for AnywhereAmps (apart from the cylindrical housing) within a single piece of plywood, having the machine do the work and, after a while, just minimal finalizing work for cutting and sanding the parts.

Plus, if I want another prototype or have other speaker diameters: Adjust some parameters within my CAD tool, rebuild tool paths, send to the machine’s computer, clamp a fresh piece of plywood and hit “run job”. Done (in the ideal world).

On the other hand, the software tools process takes its time, too. Find my introduction article on FreeCAD linked here as an example.

Spend more money on portal hardware or rather on spindle drive?

This is a question to the experts and I’ll only be able to answer it myself after a while of owning a CNC. I chose to spend more money on the portal and less on the spindle for a couple of gut-feeling reasons:

The portal is my machine, the spindle just a component of it.
The spindle is cheaper, more easy to exchange, and by far less work to mount than the portal so I’d want the portal to be a constant
If I’m limited by the capabilities of my portal milling machine, I’d have to first figure out why, and then fiddle with individual components to improve. If the spindle is the limiting factor, I’ll just buy a more powerful one.

How much spindle power?

That’s my first hard one. I’m almost certain that for my woodworking stuff, I don’t need a 2.2kW monster. To get decent feed rates and to maintain enough headroom for density inconsistencies like branch forks, it might be a good idea to orientate in the hand-router wattage segment between 700W and 1200W which always served me well. I chose a 1000W universal motor machine.

In contrast to their three-phase induction machine counterparts often called “HF-Spindles” (HF = High Frequency), they don’t require a variable frequency drive and external cooling as their fan is bolt-on their shaft. On the other hand, they tend to be louder (the fan is not independent off the motor’s RPM) and less efficient (fan losses, commutation losses, copper losses, magnetization losses etc.) I found only rough figures on the internet when it comes to efficiency, so let’s assume:

1000W universal motor drive *30% total efficiency incl. Electronics & fan: 300W @ motor shaft

700W asynchronous drive *85% efficiency *90% VFD efficiency *97% external cooling: 520W @ motor shaft

This would be a huge difference still, so an upgrade at a later point in time might be worth after all. Plus, lower wattage ratings will suffice for a HF-spindle compared to a conventional drive.

Automatic Tool Changer - yes or no?

It depends on what I need, I guess. I’m buying a simple AC hand-router with digital speed control as a spindle first. A “proper” tool changer - proper meaning that I can also mount tool shafts of 10mm and bigger, have balanced collets, 3-Phase drive with stand-alone cooling and variable frequency drive - would cost ~10 times as much. Plus, each ER20 collet chuck for the tool changer easily comes at 1/2 price of the simple AC hand-router cost. Of which I’d require more than one if I want the tool changer to make sense. And I’d need a compressed air equipment with control box etc. The total of this would easily outweigh the portal milling machine’s cost!

Conclusion: If I’ll be using the CNC very often in the future and if my projects absolutely require frequent tool changes for roughing, smoothing, chamfers, 3D etc., then I might consider this expensive step. Maybe. In the distant future.

Which size is best?

Ideally, for working with plywood sheets, it could load a full standard sheet of 1250mm x 2500mm or a fraction of it, e.g. 610mm x 1250mm. But even the smaller ones wouldn’t fit my tiny flat’s cellar. So I chose one with 840mm x 740mm clamping area.

For your choice: Check out if you’re constrained in space. If not, then thoroughly make up your mind about what you want to do with your machine. The machine size will fall into place automatically when you know your primary use cases.

plug-and-play or an assembly kit?

This depends on multiple aspects: Your budget, time you’d like to invest, and of course, if you’re at least a bit talented with screwdrivers, nut runners, cable strippers, and the soldering iron. It is also a matter of if you’d like to or need to know every bolt of your machine.

ball screws for moving the axes?

Do you want rapid moves, high repeatability due to reduced or eliminated play, or higher feed rates? Do you want climb milling for improved surface quality and longer tool life? If any of these is a definitive yes, then you’ll have to spend the extra money for ball screw axis drives.

additional versatility?

As described in the wait, what? section, the portal itself would also be suitable for other purposes. There are dragknifes for plotting, 3D print heads, and even engraving lasers available that could be fit to the portal if needed.

To prepare for this, a spindle that fits the euro-standard 43mm diameter neck holder possibly is the best choice because most other tools are available for this fit as well, making a conversion more easy.

General

Does it create lots of dust?

Yes. Get a proper dust collection. I’d recommend a cyclone system or one of these huge textile bag collection thingies that are usually used with table saws. If you’re using your machine regularly, even a shop vacuum alone might not have the holding capacity you require.

You should also ground it well (e.g. by connecting its ground wire(s) to your switchbox’s ground point) and buy antistatic vacuum hoses which you can connect to the vacuum’s or the dust collector’s earth. This way, you avoid getting bzzz’ed everytime you touch vacuum-related things plus you can exclude one hard to spot reason for your machine going mad, e.g. sudden software freezes, control randomly detecting Emergency Stop events, phantom reference switch clicking or even stepper pulse hiccups. There are many appearances in diverse forums where this was or could have been the culprit, e.g. here.

Is it loud?

Yes. The sound level is comparable to a hand router. It is loud in idle already, and it gets worse when you plunge through material. So it might be a good idea to either buy the quieter but more expensive high-frequency spindles with external cooling. In addition, your machine should (and according to your local regulations even might have to) get an enclosure for safety reasons, but also for additional noise reduction. It will also help reduce fine particle dust.

But sound also heavily depends on the bit you’re using. In my experience, large diameter, 10mm shaft, short bits sound more pleasant and are maybe even less noisy than smaller diameter or long cutters.

What is the workflow?

Here’s a simplified image of the workflow with a CNC machine.

Similar to hand-crafting, it all starts with an idea. Parts drawings and dimensioning are made in a Computer Aided Design tool instead of on a blank sheet of paper. The drawing or 3D model file is then further proessed in a Computer Aided Manufacturing tool, where suitable router tools are selected, the milling strategies are defined, and the toolpaths generated. The resulting G-Code file is then sent to the actual CNC software which controls the stepper motor drivers. They are sending current pulses to the stepper motors which then mechanically move the machine to create the part.

Operational

conventional or climb milling?

This is a choice you make in your CAM tool.

Conventional milling = the router bit “carves” the material, its cutting edges rotate against the direction of material transport. Table saws e.g. always use this to avoid kick backs. Climb milling = the router bit “eats” into the material, its cutting edges rotate with the direction of transport. The cutter tends to pull itself into the material. More reference e.g. on harveryperformance or tormach’s blog.

Which of the two you should choose depends on your machine’s stiffness and weight, your stepper motor’s holding capabilities, the power of your stepper drivers, and the material you’re working with. If you have a stiff, heavy, powerful machine (with ball screws) and work with materials on the soft side, use climb milling. Watch your machine closely to see if all feels and sounds (!) smooth. If in doubt, use conventional milling. For hand-operated routers, always use conventional milling!

How to create easy and quick workholding?

This again depends on your stock:

Big Sheets: Vaccuum table or guard fence with clamps that pushes your sheet against the fence
Small sheets: Clamps or bolt-down to table
Small blocks: Vise
Irregular shapes: Clamps that force the stock down to the working table
Dual side milling: Fixtures you should mill and glue the stock to for this specific project

Hmm, the spoilboard with guard fence will likely be one of my first projects with the new machine.

Which bits / end mills to start with?

I’d recommend the following general mills for a start. They should of course match your material in their making, spiral type, flute count etc.

Classic 6mm or 1/4" upcut end mill
2mm upcut end mill for fine stuff or small holes
90° engraving mill for chamfers and engravings
Face mill of your choice for huge surface milling
ball-nose mill for 3D smoothing

My focus is plywood, what are suitable feeds and speeds?

Birch plywood is hard but forgiving. Suitable values always take into account machine stability, power and spindle performance.

Remember: You want your bit to cut chips, not create dust. That’s why you should avoid low feed rates at high spindle RPM. For starters in wood: use half of your tool diameter as depth per pass, calculate the spindle RPM using feeds & speeds formula or a calculator from the internet, and try these out. Look at the chips. Listen to the machine while working. Does it sound “good”? Increase depth per pass or feed rate gradually, stop at where either your spindle sounds stressed, your finish deteriorates, or you feel less confident with your machine. Turn back again just a notch, and that might be your ideal feed rate & depth per pass settings for material and mill you’re currently using.

For my machine, I confidently chose F 3500mm/min = 138in/m at S 24000 RPM and a depth increment of the mill’s diameter Z 6mm, and lane-by-lane feed of 90%. For your reference, I used the websites cnccookbook, cnc-rechner, and tool manufacturer’s websites like Sorotec’s to calculate mine.

Continue reading?

CNC machine: the build CNC machine: electronics

Inrush Current Limiter - Part2

Schallbert — Thu, 10 Feb 2022

In this article, I will concentrate on two ways to create a solid-state inrush current limiter circuit: One copies the relay-approach from Electronics knowledge: Inrush current limiters, just exchanging the relay with a p-channel MOSFET, and adding a more defined time delay. The other one has a “linear” approach turning power on gradually using a PNP power transistor.

Motivation: Inrush current limiter for USB-PD

As my experiments with USB-PD for AnywhereAmps failed, I have to create a circuit that limits inrush current to my amplifier before trying again.

Circuits explained

The two circuits are very similar. They only really differ in two components of the power section. Generally speaking, increasing delay capacitor C4 or increasing charge resistor R9 will prolong the delay time.

Input section (blue)

The input section features a transient voltage suppressor diode to eliminate reverse current or overvoltage cases.

Delay section (yellow)

When switched on, C4 will gradually charge via R9. When the NPN bipolar transistor Q2’s threshold voltage is reached, it will slowly reduce voltage behind R11 which will in turn activate M2, the power path of this circuit. R12 has been added to provide a stable operating point for Q2.

Power section (green)

The linear variant has a PNP power transistor Q1 in the main current path (image below). It is activated via Q2 and the current it can provide is controlled using R10, which in this case has a relatively low value to allow high emitter-collector currents.

While the switching variant uses a current limiting resistor R8 that is shorted by the p-channel mosfet M2, it’s voltage-controlled nature does not require special care when selecting R10. You should just make sure that it’s (negative) gate threshold voltage is well met.

Simulated load (purple)

My load is the power amplifier circuitry. To simulate its idle state and the powering-up behavior, I chose a 400Ohm resistor R5 and a 1200uF polarized capacitor C3. The TVS diode protects from over voltage and reverse currents. I have tried the simulations with capacitor values of up to 10000uF and resistors down to 40Ohm, all with similar and good results.

Circuit comparison

The blue line in the diagram is what is most relevant: It describes the output voltage of the system over time. Both circuits are tuned to reach target output voltage after 0.2 seconds. Still the curves differ quite a bit showing the different nature of the two approaches.

P-CH MOSFET approach

Due to switching operation, inrush current limiter is effective but not exactly smooth
power losses are very small for this variant because both triggering NPN transistor and power mosfet don’t use much power on their own.
wide range of usability for connected devices only limited by the mosfet’s maximum continuous current.

PNP power transistor approach

Very smooth operation, turn-on time can be controlled very well because of the PNP’s large ohmic region
higher power losses when turned on due to high base current through R10
Not very versatile, as high load currents will create a considerable voltage drop over Q1’s emitter-collector path.

Further improvements

To keep the versatility of this circuit high while at the same time keeping a smooth voltage curve, one could use an operational amplifier to monitor load current, regulating it down to allowed values if needed. The power mosfet could be kept while the disadvantage of the narrow ohmic operation band would vanish. The cost would be additional complexity like described in this electronics stackexchange post.

USB Power Delivery

Schallbert — Thu, 27 Jan 2022

USB Power Delivery

USB Power Delivery is a standard within the USB 3.x protocol that specifies power supply and voltage levels within the USB network. With USB-PD it is possible to daisy-chain consumers which then negotiate power demands with the source(s) automatically. The standard has different versions, of which the latest (2021) allows up to 5 Ampères at 3.3…48V and a maximum of 240 Watts of power.

USB-PD roles

In the USB Power Delivery Standard, there are different roles that participants on the ends of an USB cable can play. They can be host or client for data purposes, and at the same time power source, power sink, or both for other participants.

Role terminology

USB-C uses the acronym DFP or Down-Facing Port to make clear that the port is the one of a Host. UFP or Up-Facing Port is used for clients that can be connected to hosts. There’a third role, though: DRP or Dual-Role Port. This port can serve either as a host or as a client, depending on who the connecting partner on the other side of the cable is and what the USB network negotiates.

In USB-PD context, any of those ports can be configured to be sinking or sourcing current. Example: Although a Laptop’s USB-C port’s role may be configured to DFP, when connected to a power source, it can still be a power sink. Hmm.

In this post, I’m only using the UFP role in both USB-C and USB-PD word sense, meaning that I’m trying to configure a power sink for my device so that it will get the input voltage and current it needs.

USB-PD version

Before USB-PD Standard 2.0, USB-C devices only support so-called Power Profiles with fixed voltages of 5V, 12V, 20V and currents of up to 2A, 3A, or 5A.
Between USB-PD Standard 2.0 and USB-PD 3.0, Power Delivery Objects replaced the old Power Profiles that allow different voltages/currents to be selected by the sink depending on the power source’s offerings. Voltages are fixed to 5V, 9V, 15V, 20V with current limits of 2.25A, 3A, 5A depending on source capability and voltage setting. PDOs allow fallback strategies, so if a source cannot support the highest power demand, the sink may select another PDO with less voltage/current that works for both source and sink.
With USB-PD Standard 3.0 PPS , the protocol offered even more flexibility with Programmable Power Supply. Here, a power sink requests a voltage level between 3.3V and 21V, selectable in 20mV steps and 0.2 to 5A, selectable in 50mA steps while staying fully backwards-compatible to USB-PD 2.0’s Power Delivery Objects.
Latest USB-PD Standard 3.1 even enhances Programmable Power Supplies’ limits and allows 3.3V - 48V and up to 5A using AVS - Adaptable Voltage Files. Whow.

Not all sources (wall plugs, powerbanks etc.) can supply all Voltage levels or currents, or even share the same standard of implementation. But a number of values is pretty common: 5V, 9V, 12V, 15V, 20V. Some of the products I have seen on the market offer up to 4A, but most keep it with up to 3A and voltages up to 15V are most common (in 2021).

Power delivery ASICs

ASIC stands for application specific integrated circuit, and in this post I’m trying to shed some light on ASICs with Power Delivery sink role that can work stand-alone, that is without an external control device such as a microprocessor. I found three types of these devices:

Standalone USB-PD sink, voltage and current setting via resistor cascade
Standalone USB-PD sink, voltage and current map programmable via data bus (I2C/UART/SPI) into non-volatile memory of the ASIC
Standalone USB-PD sink with integrated microprocessor that needs to be programmed in order to work properly

What all ASICs have in common: They require peripheral circuitry to work - at least some resistors, capacitors, and power transistors to switch power supply and negotiate with connected power sources.

USB-PD is complex: Versioning and power contract negotiations

Both the downwards compatibility and the scaleability of USB-PD take their tribute - the USB-PD specification is a >600 page monster and it’s not always clear which source or sink supports which protocol level.

As both the source, sink, and the cable have to negotiate a Power Delivery contract based on the sink’s requirements, the source’s power options, and the cable’s capabilities, you can imagine that this process is very complex. Even more so as all participants may have implemented a different USB-PD standard, revision and version. Fun fact: The USB Power Delivery specification package’s zip folder on the official page has a file size of >42MB. Although I think this topic is very interesting, I don’t have the time to deep-dive into the specifics 😅

So, let’s leave all this stuff behind and try finding an easy-to-implement solution.

Example ASICs

I have arbitrarily chosen three ICs from the web, supplied by different companies, one each per device type:

Cypress/Infinion’s CYPD3177, a resistor-config USB-PD sink controller
STMicroelectronics’ STUSB4500, a Memory-config USB-PD sink controller
Microchip’s UPD301C, a fully programmable USB-PD dual role controller

I’ll select the STUSB4500 first.

Chip / IC Shortage

For integration into my design, I wanted to buy some samples. Unfortunately, Chip shortages hit hard so that neither of the distributors that I normally order electronics parts with, had any in stock for the next year.

That’s why I decided to order a development board with the ASIC readily mounted. This is 10x more expensive than the chip itself but I didn’t have a choice here.

Integration of STUSB4500

STUSB4500 is using the USB-PD Standard 2.0 - so not the latest greatest one - but downwards compatibility of later standards make it a quick and easy starting point for a maker. The device comes pre-programmed with three Power Delivery Objects:

PDO1 = 5V/1.5A
PDO2 = 15V/1.5A
PDO3 = 20V/1A

So for my AnywhereAmps Alpha 1.1 evolution 1.2 I can use this thing right out of the box without re-programming. If I wanted to change the Power Delivery Objects, I’d use an Arduino board with I2C and install STUSB4500 library which is readily available e.g. at Platformio. With this, I could even activate USB-PD 3.0’s Programmable Power Supply if I wanted. Let’s see, when I find the time, I’ll prepare a specific blog entry for programming STUSB4500.

Powering up the amplifier

OK, so I connected the eval board to a 15V capable 30V USB-C power supply. The purple light indicates 15V available on the output rail.

Now I attached my amplifier circuits at its output terminals for an initial test. STUSB evaluation board lights gone, amp off. Meh. At least I have an indicaton for what the cause might be: It’s capacitive inrush currents, again! To be able to properly power devices with large bulk capacitors, either I’ll have to set current in the PDO to abormally high levels or I make up my mind aboud constructing a proper, miniaturized inrush limiter circuit. I have setup a prototype already that I discuss here.

Should you use USB-PD for your projects?

The answer is a definete “it depends”. Although the capabilities of USB-PD are great, there are a few drawbacks:

requires an extra ASIC with periphery to negotiate the power contracts. This takes PCB space and increases project costs significantly
To be fully backwards compatible, it makes sense to buy an ASIC that both supports Standard 3.x, and ideally, you take one that is programmable
If you take a programmable device, there’s an additional learning curve for software development, the ASIC-specific libraries you want to use, and possibly even the need for programmer hardware
You have to make sure that power supply and cable are compatible to your USB-PD sink’s needs. Cables for high currents are expensive, too
USB-C port is miniaturized. SMD soldering and entry-level soldering stations might not become friends ever, so you should bring some experience on that topic.
Barrel-jacks are just round and thus much easier to mount in an enclosure than a PCB with attached USB-C socket that also has to withstand considerable physical strains.

On the other hand, using USB-PD makes much sense when you

have a mobile gadget that you want to take along with you, or that is often connected to and disconnected from power sources
want the flexibility of battery-powered devices: Just connect a compatible power bank and off you go
need a data connection to your device anyways
have special constraints on your power supply and want very specific voltage/currents to be delivered
want easy overcurrent and overvoltage protection out of the box

Metabo / Cordless Alliance Systems

Schallbert — Thu, 09 Dec 2021

Power Source for AnywhereAmps

Status quo

I have used Metabo / CAS (Cordless Alliance System) batteries for both AnywhereAmps Alpha 1.0 and Alpha 1.1. The latter already uses the power supervisor IC MAX17562, protecting both battery and circuits from reverse current, overcurrent, and undervoltage.

Problem

Although using the power supervisor IC was a big leap forward, still it would be better if I was able to read battery state of charge directly. This wouldn’t just enable safe operation, but also allow auto-switching the amplifier off when the battery goes low without side effects like auto-restart when battery voltage jumps back up when load current drops on shutdown.

Discussion with Metabo/CAS

Metabo logo ^[1]

That’s why I contacted the Metabo CAS team, asking whether I could get some support and the specifications for communicating with the battery pack. I was quite happy to see that they respond really quickly, and soon I was on a phone call discussing the matter.

Outcome:

Usage of CAS batteries in other devices than the alliance’s power tool families is considered a “hack” and thus wouldn’t be approved by Metabo
Metabo is interested in cordless tools manufacturers to join their battery alliance, but my application in an amplifier is just too far away (Metabo’s slogan once was “Work, don’t play”)
Metabo would expect me to produce 10k+ units per year before I could joind the alliance - which is just unrealistic
I’d have to understand, but sharing the specifications and allowing me to use the interfaces would draw some of their resources while they wouldn’t have any expectations on return of invest

As disappointing 😔 as this is, it didn’t come totally unexpected. The engineer on the phone made time for me. He told me that they’d appreciate my enthusiasm about their battery systems very much and that I shouldn’t feel discouraged.

Well. Let’s talk about an

Alternative

I know only one other power standard also featuring battery powered systems that meets my expectations below.

Requirements

“open enough” for my usage
safe
of high quality
meets my purpose’s voltage and current demands
widely available

USB-PD logo ^[2]

USB Power Delivery

The standard is called USB Power Delivery. Although it went through my mind earlier, I never wanted to actually use this because it involves massive additional work.

USB Power Delivery uses a complex protocol to establish a power contract between source and sink devices. All this has to be developed, a specialized IC has to be selected and peripheral circuits need to be designed for this to make it work with my setup.

😱 Oh no, they are out of stock!

I’ll write a separate blog post once I got a grip on this new subject.

Configure MAX1756x for your application

Schallbert — Sat, 04 Dec 2021

Power Supervisor configuration made simple

I use MAX17562 ICs in my design for AnywhereAmps Alpha 1.1. For you, I made a quick summary of MAX1756x features and how to configure them. 🙂

Of course, the datasheet is the master of all information I provide below. I’ll just add some design considerations, details on my way of using it, and provide application proposals.

What are they, again?

MAX1756x is an adjustable overvoltage/ undervoltage and overcurrent protector IC. It is available in three different versions for handling over current events:

MAX17561: Will turn off in an overcurrent event and autoretry after typically 600ms
MAX17562: Will latch off in an overcurrent event; restart after enable cycle
MAX17563: Will continue normal operation with current limiter active (dissipates heat)

Both versions are limiting overcurrent events to the configured current limit.

Use my MAX1756x application tool!

To simplify usage of Max1756x IC in your application, you may use my Max1756x application calculator spreadsheet.¹

MAX1756x current limiter / overload detector

Courtesy of Maxim Integrated: SETI resistor config ^[1]

The IC features an adjustable current limiter and overcurrent detection (SETI). When current increases above SETI threshold, it is limited to the set value. For MAX17561 and MAX17562: In case this condition remains true for more than typically 20.7ms blanking time, the device will turn off the load circuit. As the MAX17563 will continue normal operation, its power dissipation might lead to over temperature shutdown at some point.

This is how you configure the current limiter / overload detection using the SETI input: Select a resistor based on the curve on the right (taken from the datasheet) and connect it to SETI and GND. For AnywhereAmps Alpha 1.2, I’m using a 3k3 resistor to get a current limit of ~3.5A.

MAX1756x undervoltage configuration

The IC features undervoltage lockout (UVLO). When voltage decreases below UVLO threshold for typically 16.7ms, the device’s power transistors will switch off the load circuit. Internally, the IC will compare the input voltage at the UVLO pin with the reference voltage Vbg = 1.2V. To set the undervoltage lockout threshold, use a voltage divider to keep UVLO level above Vbg during normal operation.

MAX1756x overvoltage configuration

The IC features overvoltage lockout (OVLO). When voltage increases above OVLO threshold for typically 16.7ms, the device’s power transistors will switch off the load circuit. It works identical to the undervoltage lockout with the only difference that voltage at the OVLO pin should remain below Vbg under normal operating conditions.

MAX1756x reverse current protection

The IC’s reverse current protection can be enabled by pulling the *RIEN pin to GND. If you intend to use this feature, remember to not accidentally use polarized bypass capacitors like I did at the IC’s input as they will explode if you apply reverse voltage.

Fault *FLAG capability

MAX1756x will indicate a power fault using its *FLAG port under the following conditions:

Thermal overload
Overvoltage lockout defined through OVLO
Reverse current: when reverse current protection *RIEN has been enabled
Overload current defined throughSETI

Recommendation: Use TVS diodes

As the datasheet recommends and my experience from incidents in the past shows, I’d use 27V Transient Voltage Suppressor diodes in parallel to both MAX1756x’s input and output ports. It will protect (any inductive) load circuitry from negative voltage spikes on shutdown and the IC itself from surge voltages when connecting a live power supply. Plus, it will clamp reverse input voltage to non-harmful levels.

Recommendation 2: Use non-polarized input capacitors

To mitigate running reset issues when connected to “weak” power supplies or high-resistance input power wiring, use bigger non-polarized input bypass capacitors than you normally would (no, 100nF might not be enough!) - Especially if you drive capacitive or other loads that cause input surge currents.

For more information how to avoid problems here, please read my knowledge blog entry about inrush current limiters.

Disclaimer: Although I thorougly read the datasheet and carefully added the data, I can of course take no guarantee whatsoever for the results you yield from this tool. ↩︎

Inrush Current Limiter - Part1

Schallbert — Sun, 14 Nov 2021

Electronics knowledge article: Inrush current limiters

Your browser does not support the video tag.

Action! Using a relay based inrush current limiter.

What is it?

A so-called inrush current is the surge current that flows through a circuit directly after start-up until the circuit has reached its steady state. High inrush currents are often unwanted because they can overload circuit parts reducing their lifetime, create electromagnetic interferences, and cause a short-term voltage drop that may lead to running resets of a controlled device like described here or to other undesired behaviour.

Example

Oscillograph: inrush current of an incandescent light bulb ^[1]

Incandescent light bulb: The pink line is the power supply’s output voltage, the yellow line traces current. You can see that the power supply decreases output voltage to reduce current draw during startup. After ~100ms, the voltage stabilizes as current flow through the lamp still reduces a bit while the filament slowly reaches steady state temperature.

Background

The bulb’s filament acts like a positive temerature coefficient (PTC) resistor: It has a very low resistance at room temperature and a much higher resistance at its normal operating temperature of ~3300K. A small experiment proves this: If I take a 116W halogen incandescent light bulb, its resistance in “steady state” at 230V is around $R_{steady} = \frac{U^2}{P} = 456{Ohm}$. When I take its resistance using a multimeter in the switched-off state, I measure $R_{off} = 38.6{Ohm}$. While this lamp normally consumes $I_{steady} = 0.5A$ of current, at the moment I switch it on, it will draw about $I_{inrush} = 6A$ which is 12 times as much!

Inrush current limiter circuits

Inrush current limiter circuits are quite often used in devices with big inductive or capacitive loads. Examples are transformers, motors (inductive), power amplifiers, switching power suply (capacitive) etc.

There are several types of inrush current limiters (TI application note) with varying complexity.

Negative temperature coefficient resistor (NTC) / thermistor

A negative temperature coefficient thermistor inrush current limiter is the simplest circuit in this comparison. It consists of only one additional part. The NTC’s resistance vs. temperature behavior is opposite to the one of the above incandescent light bulb example: its resistance reduces with increasing temperature, allowing more current to flow through a circuit.

It is placed in series connection with the load. Most NTC’s resistance vs temperature behavior is logarithmic, so its resistance is by far lower at high temperatures. This way, it very effectively reduces inrush currents at the cost of additional power dissipation as the NTC will continue self-heating, keeping its low resistance. This kind of circuit is used where the additional power dissipation is not an issue, and where system complexity shall be kept low while cost efficiency is at its best.

Limiter using a Start-up relay

This circuit avoids the disadvantages of additional series resistance to a load by adding a power resistor and a relay. It is typically used in high-power DC supplies or sensitive devices like amplifiers. The relay contact is normally open and wired in parallel to the power resistor which has a value of typically 1...22Ohm depending on the load. Behind this, the load is connected which causes the inrush current to occur. For this example, I take a car’s light bulb as load. When switching the device on, the resistor will limit circuit current and voltage on the load side will gradually increase. Once the voltage at the NPN BC337 transistor’s base is high enough, it will switch on, allowing current to flow through the relay coil which then short-circuits the power resistor so that the load can unleash its full power potential.

Calculation of R2, R3 depends on the steady state voltage you get behind R1 when relay coil is off, the characteristics of the transistor you choose, and the current the relay needs to work properly. In the example I choseR2 = 1kOhm as I need the transistor to safely drive ~40mA to the relay, so I couldn’t select any higher value having BC337’s current amplification value in mind. But I didn’t want to have it switched on early, so I added R3. The higher its value, the earlier the relay will activate, but choosing its value too low will reduce BC337’s base voltage so much that it cannot even switch.

A similar circuit is used in this application note for a toroidal transformer soft start.

Here’s how the circuit looks like in real life:

I’ve taken the liberty to add a switch so controlling circuit power supply would become more comfortable to me. The relay features a freewheeling diode to protect the transistor from negative voltage spikes.

The inrush current issue is actually solved. The comparison (yellow line) shows an unregulated inrush current of 3.5A, while the limiter reduces it to 0.8A which is just slightly above nominal current. A power resistor with smaller value would have reduced this even more.

The green line indicates BC337’s base voltage. You can see that it slowly increases (Oscilloscope accuracy at 40VPP is not the best), and will turn on the relay when current flow (yellow line) has stabilized.

Soft-start circuits / configurable current limiters

There are dedicated ICs that help soft-starting a system, depending on

Wattage needs
Ramp up time
Slew rate constraints
Total current limit

In addition, there are different types of inrush current limiters to be selected based on your application:

Time-controlled: System is fully turned-on after a specified time (simple but dumb)
Voltage-controlled: System is fully switched on above a voltage trigger level (still simple, medium cost)
Current-controlled: Limits system current or switches system fully on based on current trigger level (likely more expensive)

Thus, a fitting IC is to be selected. They may be called Power Supervisors, IC soft starter, Supervisor & Reset IC or similar. They may even have internal switching power MOSFETs and use minimal peripheral circuitry. I’ve been working with MAX17562 for my evolution of AnywhereAmps.

Continue reading here: Inrush current limiters - Part2

Transient Voltage Suppressors

Schallbert — Sat, 13 Nov 2021

Electronics knowledge article: TVS diode

What is it?

A transient voltage suppressor diode protects a circuit from short-term voltage spikes. TVS diodes are available for different voltage settings, peak currents, and with uni- or bidirectional protection capability.

Differences between TVS and Zener diodes

Like Zener diodes, TVS diodes are normally operating in reverse bias direction. They also share the same electronics symbol.

But while Zener diodes are designed to work at their precisely defined breakdown voltage with a constant control current in applications where voltage regulation is needed, TVS diodes normally operate below breakdown voltage as they are protective devices, clamping overvoltage spikes to acceptable levels. They are designed to withstand high peak pulse currents.

Differences between TVS and Schottky diodes

The symbol for Schottky diodes is just slightly different to Zener/TVS diode’s. They commonly have a lower forward voltage than their silicon pendants and are known for their fast speed of switching. That’s why apart from rectifier circuits, they are often used in reverse voltage protection circuits. TVS diodes share the quick switching behavior and the low forward voltage, but unlike Schottkys they have a well defined reverse voltage threshold.

How to select the correct TVS for your application

For DC applications, a unidirectional TVS schould be fine to use. Select the Stand-off voltage of the TVS in a way that it is a little above your normal operating voltage. Then look at the clamping voltage you get further “down” the line. Is it still below your protected Circuit’s “Absolute Maximum Ratings”? If not, choose a different TVS, make some compromise with the voltage (but don’t exceed the breakdown voltage of the TVS), or use another transient suppressor device like a voltage dependent resistor or similar.

Typical TVS diode: Voltage/Current diagram ^[1]

Power supervisor MAX1756x on capacitive load

Schallbert — Tue, 02 Nov 2021

Argh, I killed parts of my Power&Preamp circuit for AnywhereAmps Alpha today. During testing.

Incident

I accidently switched VBat and GND on my PCB. You shouldn’t think this happens at all, but it did again. The power input/bypass capacitor blew up instantly but my supervisor chip protected the rest of the circuits.

Repair and tests

After repairing with a ceramic capacitor, I continued some tests.

Reverse polarity test - PASS
Hard short to GND test - PASS
Undervoltage lockout - PASS
Normal voltage retry - …

Irregularity

But then, when I raised voltage slowly to escape the undervoltage lockout the system wouldn’t turn on, but still current consumption went up. To analyze, I connected an oscilloscope to see what was going on. You can see that the power amp’s >1400µF bulk capacitance acts like a soft short to GND so that the power supply voltage drops by 2V within 50µs to the undervoltage lockout trigger level. The supervisor IC then sheds the load, which makes the supply voltage jump back up. I suppose the ringing comes from the power supply’s regulator.

After the blanking time (datasheet MAX1756x) passes, it retries to switch on and the whole process starts over again.

Incident #2

When I measured voltage at the VCC rail supplying preamp, power amp, and 9V voltage regulator during unsuccessful switch-on, I noticed that negative voltage spikes occur at around -4V when the supervisor IC switches off, caused by the inductors in the power amp supply lines. While I was still wondering that these running resets might not be good, my preamp IC exploded. Hm.

What I learned

Never use a polarized capacitor as bulk/bypass “in front of” your protective circuits!
Use larger-than-100n bypass capacitors before voltage sensing devices when switched loads are capacitive. It helps stabilizing the voltage locally.
To protect the output side of the switch, add a Schottky or transient voltage suppressor (TVS) diode that clamps potentially harmful voltages
Limit in-rush current (follow-up article) as a precaution. I’ll switch to MAX17563 for the next circuit revision. It has a programmable current limit so that input voltage doesn’t drop too low in the first place.

Protect from negative voltage spikes

For my application, a fast-switching clamp diode in Schottky design is enough because behind the MAX1756x IC, the circuit will not see overvoltage spikes but reverse voltage spikes (due to inductors in switching regulator and amp design) only. In this article, I’ll explain the Transient Voltage Suppressors in detail.

Convert videos with VLC player

Schallbert — Wed, 27 Oct 2021

Website: media creation

For this website, I have been converting media files a lot. You know, the photos I shoot are really big which leads to long page load times and screen-filling images. While it is quite easy to reduce their size, and increase JPEG compression, it is a lot harder to do so with videos.

The problem

WTF, VLC?! I’m using the popular vlc player and somehow I didn’t seem to get it right. Either my video doesn’t turn out in greyscale as I planned, the output file is empty (Use H.264, instead of H.265 to fix), the aspect ratio seems distorted, or the video suddenly has rotated which I neither wanted nor expected to happen.

Items to cover

This series of steps help me (and maybe you, too) how to make simple video conversions like

How does the filter stuff work?

Preview filters

VLC has a quick-preview feature for the most common filters in the Tools -> Effects and filters menu. This directly affects the video you’re viewing, but won’t affect your saved file. It’s just meant for you previewing and controlling the effect properties these filters will have on your video. View this VideoProc blog entry for in-depth guides - and - more importantly - many images stepping you through. For the impatient fraction: continue reading 😉

Save filtered video to file

To save a filter that you selected for your preview, you must add them to the conversion profile. As long as they remain ticked, any save with this profile will modify your video, eventually even although you unticked all changes you made to the preview filters in the past!

How to change colors in my video?

Open the video file you want to modify.
Click Tools -> Effects and filters (or press Ctrl+E)
A dialog with three tabs opens. Select the Video Effects tab.
Under Essentials, tick Image Adjust.
Make your changes and hit Save, then Close. A preview of the effect is shown in the main window.
Open the playlist view by pressing Toggle Playlist (Ctrl+L)
Right-click your video, select Save... in the context menu.
Press the wrench symbol Edit selected Profile and in the Video codec tab, select the Filters sub-tab.
Tick the Image properties filter. This will make your change permanent once you save your file.
Enter a destination file and hit start.
Don’t forget to untick the Image properties filter in your profile. Also rewind the changes you made in the Video Effects tab (step 4).

How to rotate my video?

Open the video file you want to modify.
Click Tools -> Effects and filters (or press Ctrl+E)
A dialog with three tabs opens. Select the Video Effects tab.
Under Geometry, tick Transform.
Make your changes and hit Save, then Close. A preview of the effect is shown in the main window.
Open the playlist view by pressing Toggle Playlist (Ctrl+L)
Right-click your video, select Save... in the context menu.
Press the wrench symbol Edit selected Profile and in the Video codec tab, select the Filters sub-tab.
Tick the Video transformation filter. This will make your change permanent once you save your file.
Enter a destination file and hit start.
Don’t forget to untick the Image properties filter in your profile. Also rewind the changes you made in the Video Effects tab.

How to crop my video?

Open the video file you want to modify.
Click Tools -> Effects and filters (or press Ctrl+E)
A dialog with three tabs opens. Select the Video Effects tab.
Under Crop, play around with the pixel values and watch the preview window carefully until you’re happy with the crop. Note or memorize the settings you’d like to apply. Close the dialogue.
Click Tools -> Preferences (or press Ctrl+P)
On the Interface (start) Tab, select the radioButton Show Settings - All.
Scroll down to the Video element and open Filters.
Select the Video cropping filter.
find the Cropadd element within the filters list on the left. Enter the values you memorized/ noted down in the preview. Hit Save.
Open the playlist view by pressing Toggle Playlist (Ctrl+L). Then Right-click your video, select Save... in the context menu.
Press the wrench symbol Edit selected Profile and in the Video codec tab, select the Filters sub-tab.
Tick the Video crop filter. This will make your change permanent once you save your file.
Enter a destination file and hit start.
Don’t forget to untick the Video crop filter in your profile. Also rewind the changes you made in the Filters preferences item.

How to trim my video?

Note: A more detailed version of this can be found in this post.

Open the video file you want to modify.
Show Record button in VLC Player (tick View->Advanced Controls)
Jump to where you want the trimmed video to begin.
Press the red record button
Play the video sequence. Stop where you want to end your trim.
VLC’s standard location for video output: C:\Users\Username\Video

How to change video resolution?

Open the video file you want to modify.
Open the playlist view by pressing Toggle Playlist (Ctrl+L). Then Right-click your video, select Save... in the context menu.
Press the wrench symbol Edit selected Profile and in the Video codec tab, select the Resolution sub-tab.
Enter the video size you’d like to convert to and hit Save.

Hope this helps!

My Printed Circuit Boards arrived!

Schallbert — Sat, 16 Oct 2021

Sequel of Printed Circuit Board design blog entry

Arrival

After less than two weeks, my PCBs finally arrived along with all the parts needed to solder. I think they look beautiful :)

Soldering: Step-by-step

I decided to solder the PCB in intervals, starting with the power management section. This way, I can test and correct the sections independently from each other. The SOIC (small outline integrated circuit) outline can still be hand-soldered easily when using lots of flux and a tool to “zoom in” for optical inspection. The only issue was that the IC has an “exposed pad”, i.e. there’s a ground pane on the bottom of the IC for better heat conduction. Soldering this was a nightmare. Finally I managed to fiddle solder through one of the bottom vias while keeping the iron at another via vor nearly a minute until it finally worked out fine. I’m pretty happy with the result, though. The system can be activated pulling *HVEN low, lighting up the LED. Tests for undervoltage/short circuit behavior went very well - the battery pack is protected now.

The Preamp section was easy to solder as well. Only the polarised capacitors proved to be so big that they covered much of the solder pad - this is a problem because the solder wouldn’t want to connect both cap and pad but just stick to one of these.

The 9V buck converter section - well… I didn’t manage to solder it by hand at all. Quadruple Flat Packs without leads (QFN) are hard enough, but this one has an exposed pad as well and I had to prepare much ground area to get the heat away from this component, so I couldn’t even get the temperatues high enough. So I borrowed a Hot Air station, and after some anxious trying and drowning the components in flux, I finally did it.

I used my digital camera with a macro objective to inspect the solder connections.

Working

I build a lowpass and a highpass into the design to support 2-way speaker systems. To test the three preamp outputs (fullrange, lowpass, highpass), I directly connected the preamp to my speaker. It is not a good idea to do this, because the impedance of the speaker is far lower than the one of the OP-amp, so the output power is really low. Plus, the lower frequencies consume most power so the sound you hear is “high-pass” already and the low-pass volume is considerably less. Still, it works and I can adjust the gain as intended.

Your browser does not support the video tag.

Preamp test

That’s it, my design is working! And remember, it is open source so you can download the project from my Github repository and order your own copy.

Lessons learned

Here’s the pitfalls and design flaws that should be corrected for Revision2:

Prefer to place all components in the same direction (e.g. horizontally)
Use thermal reliefs as much as possible to mitigate soldering issues due to thermal conduction
Exposed pads: When hand soldering, the according solder pads must be extended to the sides of the IC for better solder heat conduction
Avoid QFN footprints when hand soldering. They are a pain.
If possible, don’t use footprint with “exposed pad” or “power pad”. A hot air station might be mandatory.
Try to leave more space between component pads, esp. if placed horizontally.
As SMD caps have more height, allow for bigger solder pads.

FreeCAD modeling workflow

Schallbert — Tue, 12 Oct 2021

Motivation

FreeCad logo ^[1]

I just received a second order for AnywhereAmps Alpha. My initial prototype already has a new home, so I’ll be building at least three more copies. All the woodworks done by hand would take 4-5h for each copy, so I decided I want to limit the woodworking stuff to the actual building-it-together process that - hopefully - only takes an hour. I’d like to achieve this by having the custom wood parts milled with a computerized numerical control (CNC) machine.

Tools, tools, tools

Such a machine wouldn’t eat the plans I scetched by hand. So I have to familiarize myself with software that is able to transform 2D scetches into a 3D model, and finally another tool to translate the components of the 3D model into paths for the CNC machine to cut my parts. I chose the FreeCAD software to do the 3D work.

Every beginning is hard.

But FreeCAD… well, I’ll admit that I’m not a CAD designer or mechanical engineer. But still, this tool is hard to get started with. Luckily, there are some quite good tutorials out there like this: youtube link but to be warned, I needed a couple of hours before I had drawn my first part. Plus, what you can see on the image on the right is still a long way before I was done.

FreeCAD workflow

FreeCAD will not tell you how to best work with it. It is a very versatile tool that fulfils many purposes. As I want to use it for drawing 3D parts and/or for G-code generation, I’ll only need a handful of its workbenches.

Of course, the following lines cannot be more than a quick start guide. There are tons of helpful materials out there like a FreeCad wiki, a forum, and many, many tutorial videos etc. at your hands that explain things better and in more detail than I can ;)

Dimensions first - Spreadsheet workbench

Let’s create a central place where we keep all our project’s dimensions. We’ll link them in the drawings, 3D models, and assemblies later so that changing values in the spreadsheet will automatically propagate.

Select the “spreadsheets” workbench. Create a spreadsheet and rename it e.g. to Dimensions.
Draw a chart with your dimension’s names. I found it well readable to have a part_description_attribute naming convention for every dimension.
After entering your dimension’s value, right-click its cell and select “Properties”.
In the following dialogue, select the “Alias” tab and enter the dimensionName you entered before
In any scetch, drawing, parts design or similar workbench, you’ll be able to use these dimensions under their Alias name. Click “Save”.

Sketch it! - scetcher in Parts Design workbench

It’s time to create a first sketch of your part using the dimensions you just created. You’ll be drawing so-called master scetches fist which contain all elements of a drawing at a specific view, e.g. on the XY-plane.

Select the “Part Design” workbench.
Create a new body using the blue lego-ish symbol.
Create a new scetch for this body using the scetch symbol. Select e.g. the XY-plane.
Rename your scetch so it’s easy for you to remember its purpose. As your first scetches are the master ones, I propose to name them master_yourPartName_xy
Double-click the sketch and start drawing your shapes.
Make sure you fully constrain (youtube link) your scetch. Use the dimensions you entered in the spreadsheet to help: Double-click your dimensional constraint and type = or press the tiny round fx button. Enter the name of your spreadsheet (there is auto-completion, luckily) and then the alias of the dimension you want to constraint to reflect.
Repeat for any dimensional constraint.

With your completed master sketches (one each for every dimension), all information is available for any third party service to mill your parts. If you wanted to do that, you could export your sketches to a dxf file here and you’d be done. Else: please continue reading.

Create 3D parts - Parts Design workbench

To create an actual 3D part from your master sketches, you’ll have to create derivative sketches that can be extrudes or cutouts. You’ll have to understand FreeCAD’s way of tree-aligning parts and scetches first: Every pad has an underlying scetch that defines how it looks like. Its extruding direction directly depends on the sketch’s plane. Each part can have one or more datum planes where other parts can be attached to, again forming pads, holes, or pockets on that specific face.

FreeCAD will only display the “last” member of your part’s tree, containing all dependant parts you designed before. To toggle visibility, simply hit the space key.

Create a new sketch in your part’s tree structure. This will be your first pad, i.e. your actual 3D part.
Choose the create an edge linked to an external geometry tool (hotkey X) and select the basic pad of your originating master sketch.
Draw your shape to match this shape and set its constraints, again using the Dimensions spreadsheet you created earlier. You shouldn’t need this too often though, as you linked all possible edges you could find to your master scetch before.
Now comes the fun part: close the sketch, then hit the pad a selected sketch icon and enter your dimensions.
Done!
To create another pad, a pocket or a hole on top of one your pad’s surfaces (called “faces”), first create a new sketch and name it meaningfully. Then click the face you want this scetch to be based on. Finally, choose the “set the Support of a scetch” tool and select the scetch you created before. Repeat starting at step 2. until your part is complete.

Hm, your new pad sits right on the master scetch so you cannot see them any more? Don’t panic, just continue reading.

Arrange 3D parts - Draft workbench

In my opinion, it’s a pain to rearrange parts in the Parts Design workbench. It’s getting much easier if you use the “Draft” workbench for this task. You can clone parts easily here as well and arrange them as you deem fit.

Select the “Draft” workbench
Choose the pad/part you have been working on before in the parts tree.
Click the sheep icon to clone your part.
Make the original pad invisible by hitting space key with the part active.
Choose the cloned part and rename it meaningfully, e.g. myCoolThing_part.
Select the “move” icon (marked green in above image).

A move consists of two steps: first you have to enter the reference location for your move. Either select the point by clicking somewhere in the 3D view and then moveing the cursor a bit (my recommendation) or enter the reference in the fields on the left, confirming each axis with Enter. Second, is the actual move vs your reference point. I recommend to enter those values using your keyboard because it’s incredibly hard to move to an exact location using the mouse. Confirm with Enter, and the move should be complete.

Cool, you now have a dimensions sheet, master sketches that fully specify your parts, the parts themselves as 3D bodies and all properly that’s neatly sorted in your view!

TechDraw: Create a classic “Paper 2D” drawing as PDF

Now, maybe you want to export classic 2D drawings of your part? Luckily, that’s an easy task!

Select the TechDraw workbench
Press the Insert default page icon.
In the parts tree, select the 3D part you’d like to have drawn
Press the Insert multiple views of drawable object(s) icon and select the views you want to be present.
Add dimensions, annotations etc using the tool bar.
Use the turn view frames on/off if vertices overlap your geometries, so you can select them again.
Insert cosmetic vertices if you need to, e.g. to draw symmetry lines.

The resulting document might look like this:

Further information regarding TechDraw workbench can be found in the FreeCAD Wiki.

How to create better PCB designs

Schallbert — Tue, 21 Sep 2021

Motivation

I have my first request for a copy of AnywhereAmps Alpha. I wouldn’t want to spend another 10h in building it, so I decided to invest the time into speeding up the manufacturing process. I figured out I could save about an hour per amp if I had the circuit board produced for me. Unfortunately, switching the Circuit Board from hand cutting, milling, drilling, soldering to a PCB design cost me twice as much time compared to a full build already. And I don’t even know if this works yet. But let’s start at the beginning.

Requirements on the new PCB

The PCB shall contain

the preamp circuit as designed for AnywhereAmps Alpha
the two-way crossover circuits as designed for future AnywhereAmps Beta
a switching buck converter to provide 9V DC for the aux effects
a Power supervisory circuit protecting both battery and PCB from overload, short circuit, under voltage, reverse voltage

This way, I want to gradually professionalize the electronics. I intend to use the same PCB for any variant of AnywhereAmps. In addition, I would get rid of some external components that I had to manufacture and place by hand: Fuse, high-current capable power switch, the 9V regulator and its cooling, and some wiring.

Component selection

Buck Converter

I searched the web for a low-EMI (ElectroMagnetic Interference) switching buck converter with an output of >1A @9V. A linear regulator would have been less noisy, but at a nominal battery voltage of 18V I would have to cope with >9W of thermal losses worst case. Plus, this solution wouldn’t have been battery-saving at all. I selected a (costy) design that has all necessary components in-a-box so I just had to get the PCB layout right for EMI reduction. Add a couple of bypass capacitors and a resistor divider for the target voltage setpoint. Done! (I thought at that time. Here’s how it actually turned out)

Power Supervisor

The only protection for my amps was a fuse until now, its value set to the highest “normal” current I expected in the circuits. In the early prototype phases I killed some opamps by accidentally applying a reverse voltage. And I want to avoid to totally drain the battery pack when I forget to look at the charge indicators. So I selected a power supervisor IC that would detect any undervoltage or fault condition and protect both circuits and battery from damage.

Passive components grouping

I learned that manufacturing a PCB with many different parts would be more expensive and more prone to errors than using less different parts, even if the total number of parts increased this way. So I redesigned many values and re-did the simulation for the audio preamp with these new values. Finally, I was able to break it down from 8/5 to 3 different resistor and 3 different capacitor values!

Capacitors, again

I first used MLCC (Multilayer ceramic capacitors) in my design, both for bypass, and for the audio decoupling capacitors. After again reading the documents I pointed out in my preamp design considerations post, I realized that this capacitor design seems to have trouble with distortion on audio signals. The image below shows a comparison of different capacitor types and sizes. You see that film capacitors have the best performance and MLCCs are worst. As I wasn’t able to find film capacitors with the size I needed on a SMD footprint, and aluminium electrolytic caps were too large, I selected polarized tantalum capacitors. For the bypass uses I selected low-ESR types.

TI Audio Design Journal: Capacitor types compared ^[1]

Circuit design

I chose the open-source software KiCad as an Electronic Design Automation tool and redrew all my existing circuit plans, adding the components described above. Look how the amplifier section only fills the right half of the schematics while the left half is devoted to power supply and supervisory circuits.

I tried to get the schematics as neat as possible to make them easy to read. I used global labels a lot and added connectors for simplified mounting.

PCB design

This was the hardest part for me. I never designed a PCB for audio usage before, so I had to read a lot of guidelines before I started.

How tos and tutorials

I list some of them below for your reference:

noise on PCBs and how to avoid
how to best place bypass capacitors
ground plane or not?
how to handle different grounds for signals, power, etc.
how to optimize parts placement for soldering
avoid these pitfalls when designing PCBs
how much current can my track handle with this width?
some PCB design specifications for manufacturability I tried to apply those design guidelines the best I could.

AnywhereAmps PCB explained

The result after a couple of evenings:

I separated the PCB into three zones.

Left: the 9V switching regulator, following its reference design (datasheet) it would create a current circle around itself using the bypass capacitors and cancel out most of the EMI. Look at the thermal design, it has a lot of copper and quite some vias to get the heat conducted away from the die.
Center: The power terminals and supervisory circuit. I placed solder jumpers to select the nominal battery voltage for the undervoltage detection.
Right and top: Preamp and analog signal circuitry, separated by the other electronics stuff using a thick ground track on the top copper and an own ground plane on the bottom. I hope this is enough…

As I’ll hand-solder the first prototypes to check if it all works OK, I used “big” 0805 SMD components. I increased the space in between them by more than the PCB specification would demand. Unfortunately, the highly integrated power ICs were not available in “hobby soldering electronics” sizes so I’ll have to go through some pain there :(

So this is how it will look like. The PCB size will be 45mm x 69mm.

Contribute or download

All the CAD models and schematics are open source and available for download and your contributions on my dedicated Github repository.

Up next: PCB delivered and soldered

Preamp design considerations

Schallbert — Wed, 25 Aug 2021

Decoupling capacitor selection

Some tech deep-dive background information about how I selected components for my preamp circuit board.

Why decoupling is needed

By decoupling, I specifically mean direct current (DC) decoupling in this case. The preamp needs to be decoupled both from the source (the musical instrument) and the sink (the power amplifier). This has multiple reasons:

the supply voltages of the three partners may differ (e.g. 9V guitar phantom, 18V preamp, 52V power amp) and I don’t want to accidentally short-circuit one of them
Output impedance, I mean the load capacity of the output stage might be limited. If I connect a low input resistance e.g. directly to a guitar’s pickup, I might not get any signal at all because the source is too weak to power it correctly.
Noise reduction through electric coupling

Why capacitors can do the job

There are multiple ways of electrically decoupling circuits. You can use a photocoupler to totally (i.e. galvanically) separate two circuits from each other while only the signal is passed between the circuits, or - in my case - a capacitor to only let alternating currents pass, and to block constant current altogether.

How does this work?, you ask? Well, a capacitor acts a bit like a water tank. If you apply a constant voltage (pressure), the tank will flood to a certain degree. When this level is reached, there’s no flow (current) anymore, neither in nor out. But if you now vary the voltage around this static level (water pressure variation), you will measure some medium (current) flowing in and out of the tank again.

It’s the same with the capacitor. If you connect it in series to a sink, it will only pass current over when the input voltage level is changing.

Decoupling Capacitor type

Multilayer Ceramic, Polypropylene, Electrolytic…? After reading this guide from Texas Instruments, I understood this document is all about distortion. Distortion would hit the hardest on the instrument input before the preamp, because this will be amplified by both amp stages. When using an electrolytic capacitor, there should be a voltage bias applied to reduce distortion in the low frequency range. With my design, the bias voltage is 1/2 of VCC, so even higher than the 5V used in the guide - I think, I’ll be fine with a “cheap” polarized Aluminium cap on the THD+n side.

There’s another thing about MLCCs I noticed myself: With loudspeakers disconnected at high preamp gain, the power amps I’m using still play music! How’s that? The power amp board is using MLCCs which act like piezo actuators. Their vibration is amplified by the PCB they are mounted on which gives a proper sound level. Crazy. For more geeky details, have a look at EDN’s blog post or - again a Texas Instruments guide.

Capacitor size

To judge capacitor size, I had to take a couple of aspects into account:

A bigger capacitor will lower my bottom end cutoff frequency
A bigger capacitor will put more load on the input source due to charge losses
A bigger capacitor will cost more, especially the high-quality ones.
A bigger capacitor might have a larger footprint.

I wanted a lower cutoff of <20Hz for full HiFi capability. As that’s the -3dB point already, I chose it to be even lower at 15Hz calculated via $f=\frac{1}{2 \pi RC}$. With an input impedance of 10kOhm (the value of the resistor clamping to virtual ground) I yield a target input capacitor size of 1µF. With this capacitor, the input source will be considerably more affected by the 10kOhms input resistor than by the capacitor with its low series resistance (ESR) of some hundred mOhms, so the chosen value is fine.

For the output capacitor, the calculation is similar. The power stage has an input impedance of 20kOhm, so the lower cutoff frequency of 17Hz is reached wich a 0.47uF capacitor already. Anyways, to keep value differences at bay, I decided to go with the 1µF as well.

Back to Project main page

Shelly13 - Part2

Schallbert — Fri, 09 Jul 2021

Shelly13: Project status, prototype scetches

I’m still thinking a lot about Shelly13 and how it can possibly work out to have a bigger speaker, a homogenous sound with some punch, and a sealed cabinet design that can be folded.

Cabinet: Sizing and Volume

Latest designs in my head work with 12x10 and a 14x10 inch shells. It will look a bit like a cake when unfolded, with a total cabinet volume of around 38l incl. speakers and amplifier mounted inside. It will be about 75cm of total height with its feet unfolded and about 29cm high when completely folded.

Speaker(s)

While my first design shows a 10" speaker mounted at the front of the cabinet, I’ll most likely switch to a two-way design for simplified mounting, mitigating the issues about milling a huge curved surface and creating an adapter plate. So I’ll chose an 8" Bass speaker in a downfire setup and a 3" mid-high speaker at the front.

Amplification

I’m thinking about using a car amplifier that takes its signal from an active crossover with separated outputs for low and high frequencies, creating a bi-amp design with bridged terminals.

Folding mechanism

Folding will take place with the help of drawer rails mounted in between the shells. It is crucial that the cabinet is sealed air-tight when unfolded, so there has to be a ring gasket that engages when the lower wooden ring at the bottom of the 12" shell meets the top ring of the 14" shell.

Nah, this is too complex!

Ok, let’s strip it down. Leave the foldability away for the initial prototype. Reduce cabinet size instead. Concentrate on electronics and manufacurability first. Outcome of this scope change is: AnywhereAmps Alpha!

Shelly13 - Part1

Schallbert — Mon, 28 Jun 2021

After playing MobFobAmp for a while, I wanted to take another step in the evolution towards a really good portable instrument combo. With this design I’m trying to address the shortcomings of the previous prototypes:

Power output could be higher
More volume in the low lows would be nice
Folding mechanism a bit fiddly.
I’d like a one-step unfold, preferably one-handed operation possible
Mechanical stability and robustness could be improved
Sensitive components mounted on the outside requiring to be handled with care

A first design

Taking the above points into account, I figured out that I had to compromise foldability a bit to get all the sensitive components including the speaker chassis to the inside of the cabinet. A ball-shaped design would be ideal for best low frequency / size ratio, but I didn’t know how to manufacture such a design - so I went with a cylindrical approach. I’d like the speaker to “pop up” so it would be protected from mechanical damage when folded. So my first idea looks like this:

Project constraints

I cannot manufacture the cylinders myself so I’d have to contact a drums manufacturer to send some unfinished custom-size Tom-toms over to me. Even if I get those, I’ll have to cut on a curved surface which I never did before, and I’d require some 3D printing / routing to craft an adaptor plate between speaker and cylinder… Or will I just have the speaker mounted on the top/bottom? Plus, I’d have to think about how to include a more powerful amplifier into my design - there are some ideas like this Pioneer car amp or the specialized Hi-Fi amp from DIY Class D but they have very different and maybe harsh requirements on power supply…

Conclusion

You see, many open items to think about. I’ll let this sink in for some time before continuing work on this project.

Paginator broken, Favorite icon placed

Schallbert — Sun, 13 Jun 2021

Adding a “tab” icon - favicon

I was worrying about a strange error message that Jekyll threw at any page reload since some pushes: [2021-06-13 22:07:43] ERROR '/favicon.ico' not found. Some online research revealed that this is neither a Jekyll, nor a Theme error, but a missing file error. See here how I implemented the icon for this page.

Pagination, again

I found out that somehow, the pagination doesn’t even work on my page while the layouting assumes that it’s preparing the pages. Now, if I try to access the page by pressing the “previous” button, I get a 404 error. Till now, I wasn’t able to fix this because the pagination fails silently and I do not know where. Switching it off completely for now.

MobFobAmp

Schallbert — Thu, 10 Jun 2021

Plans

Plans for my Mobile Foldable Amplifier.

Front and side view scetch with dimensions

Top and bottom view scetch with dimensions, shows the mounting side of the panels

Minimal added circuitry: Switch, fuse, LED, 1k Resistor to set amp gain to max

Lists

Tools

Required tools
Power drill
Router, copy ring, rip fence
Multi-tool: Sanding, Wood cut, cutter head, compasses
Optional: hand-held circular saw with guard rail
Hand saw, fine
Clamps
Chisel, 12mm
Hammer
Screwdrivers, bits
Wood drills: 3mm, 5mm, 6mm, 8mm, 12mm, counterbore 45°, Forstner 30mm
Steel drills: 3mm, counterbore 45°
Cutter heads: 8mm straight, 12mm straight, 45° optional
Soldering iron, solder, shrinking tube, 1m of 0.75mm² cable etc.

Material list

Item	Size [mm]	Count	~ Cost [€]
Front/Back: Birch Multiplex	L 500 B 260 W 12	2	12
Top/Bottom: Birch Multiplex	L 260 B 248 W 12	2	4
Sides: Birch Multiplex	L 500 B 110 W 12	4	8
Strengthening & Mounts	L 260 B 236 W 12	1	2
Screws (preferably Torx)	12 x 3	100	5
Screws (Battery holder, handlebar)	35 x 4	4	0.20
Screws & nuts (Speaker)	15 x M4	3	0.20
Board holder	d5 x 16	14	3
Piano hinge	500 x 32	6	18
Latches	50 x 12	2	9
Bicycle inner tube (Flexible hinges & Straps)	28"	1	6
“P-Type” Foam or Rubber seal/ gasket	500 x 9 x 7	6	8
“P-Type” Foam or Rubber seal/ gasket	260 x 9 x 7	4	4
“P-Type” Foam or Rubber seal/ gasket	236 x 9 x 7	4	4
“I-Type” Foam Seal (Speaker)	600 x 9 x 2	1	2
Aluminium bar	260 x 12	4	2
Spacer (Auto-unfold feature)	do 30 di 4	4	2
Handlebar	What you prefer	1	?
Amplifier (e.g. Adafruit 20W Class-D amp)		1	20
Speaker (e.g. Visaton W 170S 4Ohm)	d170, W 81	1	35
Battery, 10V - 14V	What you prefer	1	?
Fuse, switch, LED, resistors: See Circuit diagram	What you prefer	1	?
Spare wood e.g. as mask	e.g. 300 x 300	1	0
Gum glue or hot melt (airtight gaps)	20g	1	3

The Build

Routing

Let’s first mill all parts so the gaskets for sealing the cabinet can be mounted.

Prepare your router with the 8mm straight cutter head @3mm cut depth, and add the rip fence so that the cutter’s center is at 7mm distance from the fence. Make sure to control the router thoroughly so your cuts are straight and clean.
Continue with the front and back parts, 500 x 260 x 12. Mill the long outer sides of the two boards.
Now cut the foldable sides of the cabinet. Take two of the four 500 x 110 x 12 pieces. Make the long sides face upwards. Use your router to cut the inner sides of these boards like so:
Now it’s time to mill the surrounding gaskets of the top/ bottom parts - mill all 4 of the inner sides. Refer to scetches for help.
As we’re at it, exchange the cutting head by the 12mm one. Re-adjust the rip fence so the router will mill with no distance to the fence. Cut the upper “rear” end of the top/ bottom part so that the rubber hinge can be mounted here later. If there are strands like you can see on the image below, gently sand them down with an 180 grain paper.
Cut the hole for the speaker in the front part. Use the compasses tool attached to the router or the multi tool to cut the inner ring of 148mm diameter. Three or four steps of router depth variation should be OK.
Mill a mask with the targeted outer speaker diameter into a spare piece of wood, e.g. height-of-your-copy-ring [mm] multiplex. As you will be using the copy ring to make the bevel, accomodate for its width so you will reach a 170mm final diameter. It should look like this now:
Center the mask on the 148mm cutout at the front and atttach it (e.g. use double-sided tape). Mount the copy ring to the router. Make sure you use the 12mm cutter head and target a bevel depth of no more than 8mm - The speaker has 7mm depth, add 1mm for the gasket.
Sand the edges a little. Use the multi tool to correct small bumps that might still be left. Repeat until the speaker fits in snugly.
Now cut the notches for the Latches’ counterparts. Set the router to a depth of material + gasket grove depth (so it can sit behind the gasket) and used the rip fence to match the center of the top and bottom parts. Note that you’ll cut orthogonally through the groves for the gaskets you made previously.

Hardware

The hardware is mounted before assembly of the cabinet as the gaskets’s counter pressure will increase difficulty of getting the cabinet shut tight.

Mount the latches’ counterparts at the top and bottom boards with two 3x12 screws each.
Use hot melt or gum glue to seal the metal’s edges against the cutout.
Cut a piece of 260mm length from the bicycle inner tube.
Cut this piece in half parallel to the long side, so you get two straps of 260mm length.
Take the aluminium bars. Cut 5 holes equidistantly into each bar, use the 45° counterbore so the countersunk heads will fit.
Put the strap of the tube below the aluminium bar and mount them at the top rear of the top board.
Repeat step 6 with the bottom board.
Put Front, Top, and Bottom boards together on the table so they form a big C.
Mount the latches centered to the front board, make sure they keep the top / bottom “lids” shut tightly and the top/bottom boards are at a 90° angle to the front.

Locators

Now mount the locators aka bottom beam to the top and bottom of the cabinet. They will keep the side parts in place and the cabinet stiff.

Mark the spots where locators will be placed at the side of the grove. You can find the dimensions in this section.
Use a 5mm wood drill and put some tape on to mark 8mm depth so you wouldn’t completely go through the board.
Cut the 6 holes in each of top / bottom board.
Make sure the collar of the bottom beam is at level with the board surface, not with the grove.
Remove the locators for now, gaskets will have to be mounted before.

Gaskets

They will finally make the cabinet airtight, but they’ll be under a large strain, especially if you fold and unfold the cabinet often. Some drops of silicon oil might increase their lifetime.

Make sure the groves are clean and dry so the adhesive tape doesn’t fall off.
Start with 4 gaskets for each the top/bottom part. They might be a little wider than the grove, use your fingernails to fit them in.
When finished, take a razor or a knife with a pointed blade to cut holes in the gaskets where the locators will be pushed in. Use the marks for the locator holes you made earlier as guides.
Use a hammer to gently push the locators through the gaskets into place
Finally, use a small screwdriver or a blade to raise the gasket like a short sleeve around the locators - the cabinet must be sealed airtight.
Apply the gaskets on the two sides.
Apply two gaskets per front / back board.

Auto-unfold

This is an optional feature. With this installed, the cabinet will always want to unfold by itself and you will have less hassle getting ready to play.

Take the four sides. With a Forstner bit, cut a 3mm deep circle right in the center of the insides of the side panels.
Cut a 70mm piece from the rubber tube. Cut it in half so you yield two straps.
Mount the strap on two of the sides using the 30mm spacer and a screw right within the circle you cut before.

The spacers are needed to increase the area where the rubber strap is secured against the wood. Without them, the rubber would tear around the screw.

Hinges

Mounting the hinges with the gaskets on will be difficult. You might use another pair of hands or some clamps to help you out.

Put the front panel on the table. Take the two sides with the gaskets on and have them aligned in a way that they “stand” on the gasket of the front board.
Wile pressing the sides against the front board’s gaskets (orange), put the hinges (red) in place and mark their hole’s positions to simplify mounting.
Mount the hinges. Make sure there is no gap between side and front panel.
Repeat with the other two sides and the rear panel.
Put the two halves together. They will want to move inwards due to the gasket’s pressure, so use the strengthening panel to keep the cabinet fully unfolded. I was using a piece of wood with dimensions of 236 x 27 x 12 for this step.
If you prepared the Auto-unfold feature, now is the time to pull the loose ends of the rubber strap and mount them on the opposite side.
Finally, mount the two hinges on either side of the cabinet’s outer surface. Make sure that there is no gap between the side parts.
Try the mechanism. If you fold the cabinet, it should unfold by itself in case you have the straps mounted correctly.

Top and Bottom

In this step, the locator holes are cut and the rubber hinge is mounted. This way, only two latches are needed instead of eight and the mechanical pressure on the gaskets is kept high enough.

Keep the cabinet unfolded.
Apply some paint on the locators that are mounted in the top/ bottom parts already
mark the spots to drill locator holes on top and bottom of the cabinet by aligning and then pressing unfolded cabinet shell and top/ bottom parts together
Use the 5mm drill to cut the holes both in top and bottom of the cabinet according to the marked dots, depth again 8mm.
Check if it all fits fine. Sit on it and try the latches. If the locators don’t fit 100%, take the drill and cheat a bit…
Mount the top and bottom panels using the two remaining aluminium bars (green) and the loose end of the rubber hinge (black) to the back side of the cabinet like shown in the scetch.

Working?!

Whow, it’s almost done! Folding and unfolding should work by now, and you maybe should hear the sound of this cabinet before doing the finishing work…

Mount the speaker, but don’t forget to put in the gasket first.

Three machine screws, M4 x15 with nuts will mount the speaker securely.
Configure the amplifier by soldering some jumpers. Follow the instructions from Adafruit for the analog setup. Mount the potentiometer or set the amp to 100% amplification right away following the circuit diagram
Fuse and switch might not be needed yet. Instead of a battery, connect a power supply with current regulation, turn it to 12V @ 1A to protect from short circuits.
Quick, solder a 6.3mm input jack to a couple of wires or an old instrument cable of yours and shove it into the left channel’s terminals of your amp.
Connect your instrument.
Turn the volume of your instrument up and enjoy!
Note: If your instrument is passive, i.e. has no built-in preamp, you’ll most likely need an additional preamp or an active DI box to get acceptable volume output. E.g. this thing will do the job very well, although most effects pedals also have a preamp onboard - I just used one of the latter.

Finishing the project

Look how sleek, lightweight and easy to carry your amp is when folded!

Some additional steps I took were to give the amp an adequate housing (the empty screw case…), add fuse, input jack(s), switch, LED and the battery. If you’re using an electric tool’s battery, normal cable lugs can connect the battery’s terminals to your setup. Using some spare wood and your router, it is an easy job to create a mount you can just click your battery in like I did for my design.

A handle might be a good idea for carrying, and a rubber strap could keep the cabinet in a folded position and the top/bottom parts attached to the rear when folded.

I sincerely hope you enjoyed the read / the build and that you’ll have as much fun taking this little thing anywhere to play as I do.

Schallbert

Back to Project main page

Markdown to PDF

Schallbert — Wed, 09 Jun 2021

The tool

Get a tool that can do the job. I selected the Visual Studio Code extension Yzane’s Markdown PDF. Easy to install, easy to use, good documentation. Only the images… well… wouldn’t show.

Getting the images converted, too

The relative links to the images are different between Jekyll’s interpretation and the notion from the PDF generator. I had to move the target markdown file into the root folder and add a . before my /assets to make it work. Pagination is …meh… but I’m not willing to put more work in this right now so that’s as good as it gets.

An issue with Posts not showing up?

Just upon publishing this sixth post of mine, I see that somehow only page 2 of the post overview is shown (looks like 5 posts per page) and I cannot switch to page 1 containing “latest” posts:

. Hm.

Jekyll: images

Schallbert — Mon, 07 Jun 2021

The trick to make images show both locally and on Github pages

When I added some images to my content, all was fine until I pushed to GitHub. Most images were gone and I couldn’t figure out why. Some searching revealed that Github is case sensitive and some of my image paths were still mixed case.

Changing folder names in git

Well, it still didn’t work because git didn’t recognize my renaming. I first had to create the renamed folder in another subdirectory, use git mv folder/tArGet* target to get this to a temporary place, delete tArGet and move target to the place where tArGet was before. Only this fixed the issue and now github and my locals are in sync again, luckily.

This site: Table of Contents

Schallbert — Sun, 06 Jun 2021

There is an easy solution for making the page’s table of contents stick: add toc_sticky: true to the defaults or the specific page’s front matter. How I found the keyword? By searching “toc” in the whole solution and some scrolling through the layout scetches. I modified the original site navigation in the left sidebar and exchanged it with a dummy implementation of a gallery / audio / video contents display.

ToC on the left

To understand how to create a sticky ToC on the left, please continue reading this chapter of the mySite project.

Author’s profile

I decided I only want my author_profile to be shown on pages like about, legal info and not on the typical content pages. So I wrote author_profile: false for the pages I didn’t want it to show up - with no effect. Why? I had set it in the config.yml’s type defaults but it should actually go into the frontmatter of the individual files.

Jekyll `-incremental` option

Schallbert — Thu, 03 Jun 2021

The images

I worked on my assets/images, they tend to be somewhat “big” so that page load times might be a reason to worry about. I’m using paint.net to shrink them to a size I think I can afford and save them as .jpg with compression max’ed out and some compromises on image quality. This way, most of my pictures take less than 10% or the original size in kB.

MinimalMistakes theme - Page design

Schallbert — Wed, 02 Jun 2021

The landing page

Yeah, by changing the layout type to single I was able to get a header image as type overlay in! The buttons are displayed within the image now. The image changes with display size and orientation automatically due to the visual designs defined in the _sass folder.

landing page image

Ha, found it! I was freaking out; my landing page now just wouldn’t show any image, no matter how hard I tried modifying the home.md file… Reason behind was that there was a stock index.html file that came with the template and that seems to have priority. No I just added my changes in there - works perfectly fine! By the way, the overlay image’s height is automatically selected depending on the overlay text’s contents/ height.

Easy solution: type toc: true in the front matter. Unfortunately, eventually you will loose this information as you scroll down because it won’t stick to your scrolled view.
Hard solution: Add page navigation to left sidebar. How? See here!

🎄 Breadboard Fireplace

Schallbert — Mon, 01 Jan 0001

Project stats

Difficulty: beginner 1/5
Cost: 8…25€
Time: ~1h

Abstract

This is a quick and easy project that demonstrates how to use an embedded device and some LEDs to create a warm and comfortable living room environment. Tied to a powerbank, this little thing will glow all evening and create a warm and comfy atmosphere. I borrowed parts of the idea for this project from a book called “Tiny AVR for the evil Genius” which I read when I started with embedded software design in C years ago.

Project cost varies on the components you choose. If you take an original Arduino, it will be on the upper end - but in this case you pay tribute to the hard-working developers and engineers at Arduino, creating open source solutions for all of us 🥳

The Hardware

Of course, you can use any microcontroller board that supports at least 8 digital outputs and 1 analog input. In my example I’m using an Arduino Nano replica board I had flying around somewhere. Time to upgrade to the “original”… You should have a small breadboard, a couple of wires and eight LEDs at the ready. I recommend using 5 warm-white LEDs, 2 yellow LEDs, and 1 red LED with similar brightness (lumen count) to yield colors that are similar to a real fire.

The Software

The software creates a flickering lights effect with varying update speed.

Setup

On startup, in setup() I define pins the LEDs are connected to as outputs. I make it read an open analog pin once and takes its (undefined) value as seed for the pseudo random number generator discussed later.

Loop

In the infinite loop, the program calculates a new random number and then hands this number over to the downstream functions. Here, a random number generator creates value which is then used to

determine which of the eight LEDs should be on
set the time for which this LED status shall be maintained.

Finally, the LEDs are set accordingly and the loop repeats at the beginning.

Linear Feedback Shift Register

The Random Number Generator is implemented as a so-called linear feedback shift register (LSFR) with a length of 32bit (repeats its pattern after max. 2^32 steps). This implementation uses the “Galois type” of LFSRs.

Galois LFSR implementation

void loop() 
{
 iRandNum = (iRandNum >> 1) ^ (-(iRandNum & 1u) & 0xd0000001u);
 [...]
}

There’s a lot going on in this line of code, so please let me explain it down to the core.

Essential to this sort of random number generators is the XOR operation indicated by a ^ in the center of the code line. XOR means “eXclusive OR”, which outputs logic 1 only when logic inputs A and B are different. When A and B are both 0, or both 1, this gate will output 0.

Example binary XOR:

 A B Out
0b00001111 ^ 0b00110011 = 00111100

Left to the XOR operator, the current randum number is shifted to the right (>>) by 1. All of its contents is made less significant by one bit. In numerical terms it means that a value is integer-divided by two.

Example binary SHIFT:

 A B Out numerical:
0b00001111 >> 1 = 0b00000111 15 >> 1 = 7

OK, let’s further analyze the above code line. Right to the XOR operator, there’s another operation taking place: A logic AND (&) between our current random number and 1u which means “unsigned 1”. What it does is masking all higher bits of the randum number generator, the result of this action is a simple 0 in case the number has a 0 as its least significant bit and vice versa.

Example binary AND:

 A B Out 
0b00001111 & 1 = 0b00000001

The negation at the beginning of this term means that, when the result of the before & was 0, the whole term will be 0b00000000... again. If it’s -1, the 2’s complement binary code will instead look like this: 0b1111111....

This is an easy way to apply a single boolean value to any longer data type without branching logic like using an if() clause.

And finally, there’s another logic AND between this term and a fixed bitmask 0xd0000001u. The latter are the “taps” of the LFSR, i.e. which bits of the current value are fed back into the system. Unravelling this hex value to boolean shows: 0xd = 0b1101 so the bits 32, 31, 29, and 1 are &‘ed with the current random number, leaving only their values untouched while all others are set to 0.

Galois LFSR Summary

Whew, that was a lot. It’s time for a couple of minutes break. Then please read the summary below for the beloved code line

iRandNum = (iRandNum >> 1) ^ (-(iRandNum & 1u) & 0xd0000001u);

Meaning: “If the least significant bit of the current random number is zero, then just shift the random number to the right. Else, in addition XOR with the tapped bits of the current random number.”

LFSRs are widely used in wireless communication technology, they are interesting for checksum calculations, cryptographers and can even be useful when programming games, e.g. for procedural map generation. LFSRs are easy to implement (as you can see they even may only use a single line fo code!) and don’t use many resources. Here’s an excellent blog post dealing with LFSRs in depth if you’re more the visual type (and less the logic one). On the down side, their “quality of randomness” is bad because they are deterministic. All in all I found it a very interesting topic which is why I decided to manually implement an LFSR here instead of using Arduino’s stock random() function.

Pitfall! Game scene: procedural level design with LFSR ^[1]

Possible issue

Well, if you fully understood the code line, you see what happens if the random number contains only 0s.

It will never get out of this state again 💀

That’s why you should never initialize this type of LFSR with 0.

Driving the LEDs

The rest of the software has much more lines, but also does much more boring stuff. It takes the generated 32-bit random value and runs it past the 8 LED outputs, i.e. every LED “sees” each bit of the value. Between each of these iterations, a randomized delay is used so the flickering speed is limited to a change rate we still can perceive.

The Hardware

Thankfully, the hardware is more easy to explain than the software:

An Arduino Nano board is connected to a prototyping board.
Eight LED’s anodes (+) are connected to the digital outputs D2...D9.
Their cathodes are bridged with jumpers and connected to the Arduino’s GND.

This is the simplest design I could think of to do the job.

Improvements

Each LED should get a series resistor.
- In my design, the current is only limited by the maximum current the Arduino Pin is able to drive.
- Either the LED can handle well above 40mA peak current or it will inevitably break down at some point in time.
- Resistor calculation should happen with the LED datasheet in mind as different LED colours have different voltages.
- Example resistor calculation @5V supply: Warm-White LED: 20mA @ 3.1V --> R = U/I = (Usup-Uled) / I = 1.9 / 0.02 = 95Ohm. Take 100Ohm.
Each LED could get a parallel capacitor.
- This would smooth the flickering effect.
- I could take videos in which the camera wasn’t irritated by LED flickering. That would be nice.
- Example Capacitor calculation: Average current when switched “off” 20mA, allowed voltage drop 0.6V within 1ms. Proposed Capacitance is then 20mA * 1ms / 0.6V = 33uF

You want a DIY replica?

Here you go! Breadboard Fireplace DIY instructions

1 year CNC 🎂

Schallbert — Mon, 01 Jan 0001

One year with ‘Zerspanobert’ portal milling CNC machine

It’s March 2023, and I’m hitting a year of operation with my CNC machine. It was a good year. I definitely learned a lot and had quite some fun writing articles about my progress. Of course, I wasn’t able to document and blog about each and every project that I ran during that period, so take this article as a retrospect summary of the year.

The machine

With 12 months of experience, I can say that I selected the right machine. It is Sorotec’s second-cheapest one, featuring a lot of stuff that more advanced portal milling machines have like ball screws, 3Nm steppers, 48V low voltage rail, preloaded linear guides. To keep the price tag moderate it misses higher-end features like belt drives, dual linear guides on the Z-axis, and additional stiffeners for portal, z-axis, and main frame.

That’s why you have to know which loads are borderline and pull back on those. But that’s likely true also for more advanced machines, they only raise the limits. What blocks quicker feed rates or higher depths of cut within my setting currently is the router motor and the way it is attached to the Z-axis.

The motor is clearly overwhelmed with

birch multiplex, full-slot milling, 6mm cutter 2-flute, S24000, Z+12mm, F4000
HPL, full-slot milling, 6mm cutter 2-flute S24000, Z+6.5mm, F3500
Aluminium AlMg1, full-slot milling, 6mm cutter single flute, S24000, Z+3mm, F1600

If you reduce depth of cut and/or feed rate however, it will be able to run the job very well. Machine accuracy is pretty good; the actually limiting factors here are the manual determination of XY0 visually and the non-calibratable tool length sensor, not the machine itself.

The vacuum table opened up the world of fine engravings with a V-cut. Problem was that before, workpiece warp and inaccuracies of the workbed caused engravings to be of very mixed quality and larger tolerances in width than I had anticipated. These issues are now gone, as well as the additional work of creating fixtures for small or cumbersome Workpieces. Holding and clamping got easier by a magnitude, and I could reduce Z-safety height which would speed up engravings by as much as 20%.

I focused on reducing machining time as much as possible so I wouldn’t have to spend more time than necessary monitoring the machine in the cold workshop. That’s why I deliberately brought it and the cutters to their limits in terms of rapid movement speed, feed rate, and depth of cut. When I figured out these limits, I dialed back on the parameters by 20-30% to stay safe and keep repeatability high.

Upgrades

I upgraded my CNC with a vacuum table and had to learn a lot about clamping forces and vacuum pump selection. I installed acoustics insulation to get noise level and low-frequency vibrations down. It’s hard to see in the below image, but the whole machine table’s underside has been covered in sound- and vibration absorber mats.

Also, a chip extraction system was added so I wouldn’t have all the dust in my workshop. It works flawlessly. The amount of chips produced is astonishing and I’m glad to have it installed since the early days.

I will soon add a dedicated 3-phase spindle motor to further minimize vibrations and tolerances induced into the motor mount. The machine will become a little more quiet, yet more powerful so that I may finally be able to reach the target feed rates mentioned above.

Materials

I have been working with hard wood, multiplex, MDF, acrylic (PMMA), Polystyrol, High pressure laminates, and Aluminium composites.

With all of these, I had to learn how the materials behave and with which parameters to run the endmills for high quality results. Some materials were so challenging that my machine was overloaded, with some others I had vibration issues or problems to raise the bar on quality but I always learned from these failures and improved my processes so they could be circumvented.

The image above shows an extraction hose clamp that I mounted to the Z-axis of my CNC, made of PMMA (Acrylic). I worked hard to get minimal chamfers just right and to utilize smoothing runs where I thought they could provide better surface finishes.

Software

I tweaked the machine’s macros, tuned its kinematics parameters, and even wrote a full-blown application to enable the CNC to engrave QR-codes into virtually any sheet material, providing a high rate of detection for pretty much any camera.

When my engraving projects got more and more complex, my CAM kept crashing when calculating paths for batch processing. Thus I had to do batches manually which made me dive into coordinate shift commands like G54...G59.3, G68, and G92 which will kill your workpieces if not properly understood. This helped me better understand the numerical controller and its interpreter language.

Last but not least the learning curve I had to get used to working with FreeCAD and Estlcam had to be pretty steep, or else I wouldn’t have been able to get my parts cut.

Help & support

Luckily, I always had people who were both willing and able to help. My wife was always there to support and allowed me to make time for this, and my kids were visiting me in the workshop frequently so that I didn’t feel lonely at any time. It is my wife and some great friends to whom I could talk to when work got frustrating. But I also had outstanding support from the commercial side:

Thanks to Willy and Roy at Sorotec for advice, tipps and tricks that helped me handle my machine, and to get better lifetime out of my endmills. I always enjoyed your direct phone support and quick turnaround times for cutter delivery and guarantee repairs. I also like to visit the hobbyline CNC forum for discussions and to publish and discuss my stories there to give something back to the community.

Also thanks to the EdingCNC team, especially to Iwona and Pim for your timely and in-depth responses on support tickets and controller software related questions.

Many thanks go to Jo from Stritzelberger for hours and hours of discussion about good solutions for my vacuum table and its accessories.

From vhf, I’d like to thank Mrs. Dressner for the time and frequent conversations about quality of cut, optimizations of feeds and speeds for the company’s endmills, and advice on material milling strategies.

Also thanks to Nicolas from Benezan Electronics for assisting me with analysis of a problem with and, finally, replacement of the signal breakout board.

Thanks to Mr. Leibold from Gossen Metrawatt for helping me select a good instrument for measuring voltage peaks and system power draw.

Cheers to Dominik from Spinogy with whom I had phone calls throughout well over half a year to select a proper spindle upgrade and its accessories for my machine.

Finally, thanks to Achim from CadAsCam, your friendly ear taking software improvement proposals, answers to basic questions, and your extremely quick response with software updates and fixes.

Runtime and jobs

Within the year, I ran 200 jobs with a total machine time of less than 18 hours. When counting in workpiece setup, machine calibration and so on, I’d triple that number. And that result again can be doubled easily when taking time for CAM/CAD into account, too.

In terms of time, I can’t tell exactly how many hours I devoted to the machine itself. I believe it is on par with maintaining this website, where the creation of 1 minute of “time to read” is about one hour that I need to invest.

I also spent hours and hours online researching for material parmeters, cutters, taking and giving advice in forums, investigating machine upgrades, and watching videos about how other people use their CNCs.

By the way, my current favorite (as of March-2023) is by CNC Connect (youtube link). I won’t try to compare myself to that but I’m always impressed by people and manufacturers who know their stuff.

Cost

The CNC hobby is expensive, both in terms of money and time. In my case, I invested nearly 13.000,00€ 😨 into machine, its accessories, spindle, machine table, power supply and protective equipment, lighting, numerical controller, laptop, CAD/CAM/CNC software, acoustic insulation, tools to build the machine, exhaust extraction system, personal safety equipment etc.

And with that, I didn’t even mention cost for endmills.

This is how it is: “The machine itself is not too expensive”, you think. But once you decide to buy, you enter a rabbit hole. I followed it down almost all its paths, making very few compromises on quality of equipment, and not specialising in one material which would have kept costs down.

I was intrigued by the different materials and their properties, and it was always a joy when a part turned out really good, independent off the matter it consisted of.

Failures & solutions

Here’s a list of bigger issues I experienced within the last year.

Router motor stop, intermittent operation. Ⓜ️

This issue was reproducible in hard wood and multiplex after just 8h of spindle operation. Solution was the installation of a new electronics PCB that Maffell supplied without further ado.

Mid-job router motor speed drop to minimum RPM Ⓜ️

This issue was reproducible as well and had two key causes:

Emergency switch had very little travel for triggering, so there were short contact bounces possible. The switch is mounted to the machine table, and could be triggered through machine vibrations. I fixed this by modifying the microswitch’s position in the housing so that the switch wouldn’t trigger early.
The breakout PCB that generates the analogue 0-10V signal out of the CNC’s pulse width command had a wrong operational amplifier type mounted (non-RTR). Under yet unknown circumstances (maybe contact bounce of emergency switch?), this part would experience latch-down so the router motor speed would drop to its lowest possible setting. A new signal breakout board was mounted and the issue disappeared.

CNC: “Velocity was higher than max!” ⚠️

This error appeared on a few occasions. They were all unpleasant. Once, I pushed my machine too hard (240mm/s on XY, 110mm/s on Z), which made it squeak and loose a lot of steps, adding up into the centimeter range.

Another time with less high (but still too high, retrospectively) velocity values entered, I made a mistake by commanding a rapid move G00 Xxx Yyy Zzz for all axes simultaneously which made the machine jump and crash into its mechanical limits on Z-axis. Thankfully, it was the upper limit so neither the spindle nor the machine or its bed suffered any damage. Only the stepper clutch slipped. Whew.

The reason: Maximum combined axes move pulse frequency is 125kHz. I considered it was per-axis. This is wrong. I double-checked it with the numerical controller manufacturer. I needed to readjust the axis’s speed values to stay below 125kHz combined.

To find new values, I looked at my jobs so far. I was loosing more time on Z-axis rapid movements than on the other two axes. That’s why I put emphasis on the Z-axis to keep it quick and with high acceleration while I reduced the values for X- and Y-axis. Note that Z-axis has double resolution due to its shorter ball screw threading, so you have to double its values to compare them with the other axes.

This all happened after improving manufacturing quality that I documented here, so you can consider these values are proven to yield good results, thus won’t be changed again lightly.

Running the machine into the vacuum table 💥

Well, that really upset me. I didn’t even have the table for two weeks, and already now I had damaged it. Worst thing was that I didn’t understand why.

All I wanted was to cut 18mm birch multiplex. I had the rougher do the outline but faced an issue that made me press the emergency switch when the cut already was complete (Travelling back to XY0, the cutter would have hit an obstacle). I fixed the path and reset the machine. I changed tools to do the smoothing, and hit “run”.

When I realized that the machine sounded unusual, I hit the emergency stop. But at over 4000mm/min for smoothing I wasn’t quick enough to avoid the machine run through the vacuum table.

I looked at the G92 work offset for Z-axis. It was correct. I searched in the CNC’s volatile variables. All matched my expectations. Then I started digging into the machine’s logfiles. Luckily, I made the above picture right after this happened so it wasn’t hard to figure out the correct log lines.

You can see here that tool length has been measured both for the rougher and the standard cutter. The standard cutter protrudes more - by roughly 3.4mmdifference. This is the exact depth that I was missing on my vacuum table now.

Still, I didn’t have a clue what caused the machine to not take the updated tool length into account. So I looked at the Tool Change macro.

Here you can see that variable #3501 captures whether a tool length has been taken already. This is vital because if so, you can just measure the new tool length and take the difference between the two, save that as a Z-axis offset and you’re fine to proceed the program with the new tool.

The thing is that variable #3501 is volatile as revealed during a good read of the controller’s manual. And it seems like volatile variables are not only reset when the application is closed, but also when the emergency switch is pressed.

Deducing from my prior statements, I had the following issue: The emergency switch press would wipe that variable’s value but keep the original Z0 height measurement that I did at the very beginning before using the rougher bit. So, instead of updating the height offset with the new tool to account for the new tool length, the macro was thinking that this was the first tool used and just saved its length to variable #4501 (because you can’t take any difference if you didn’t have a tool prior to the first one, consequencially).

This alone wouldn’t even have caused this incident if the newly entered cutter hadn’t protruded more from the spindle collet than the one before.

I solved this problem by adding a message into the macro when it was solely saving the tool length but not modifying the Z-axis’s offset. This way I’d at least have an indication when something is about to go wrong.

Conclusion

In summary, I will continue using the CNC. I will be learning as I go. And I will deepen my knowledge about how software interacts with mechatronics that changes physical matter like ‘Zerspanobert’ does. I’ll try to write down what I learned so others can follow my tracks if they like to.

I hope to be able to maintain good contact to other hobbyists and makers, and to the companies that I had the pleasure to work with so far (and, maybe, to add some more to my list).

Cheers! Schallbert

About

Schallbert — Mon, 01 Jan 0001

Purpose of this Website

I’m running this website on a voluntary basis. I do this because I’d like to spark others’ interest and fun in the areas of Mechanics, Electronics, and Information Technology. I focus particularly on the manufacture of items using CNC milling technologies, as well as on the setup, operation, deployment and maintenance of my server and its content. Occasional articles on electronics development and IT security round off the blog.

About ‘AI’

This website is and will remain free of content generated by ‘AI’ tools. The articles are written by hand (I think you can tell) and I have no intention of changing this. For translations, however, I do use ‘AI’-based translation tools. In the early days up to Q1 2023, this applied exclusively to German articles, as I had been running the blog in English.

Since then, however, I have been writing in German and having the articles translated into English, because ‘AI’ finds this much easier and, in my opinion, produces better results. Nevertheless, I proofread every translated post and make every effort to publish high-quality and interesting articles.

The research and everything else involved on this blog is always done manually.

Consequently, for every minute of reading time, I have to invest roughly an hour’s work.

Where I do use “AI”, however, is as a “rubber ducky” for ideas and solutions that I wouldn’t have come up with myself due to a lack of perspective. Essentially, I use it as an advanced search engine. But even this output doesn’t feature in the articles.

Who is Schallbert?

I’m Schallbert.¹ Hobbyist maker and programmer, Electronics technician, Electrical Engineer. I have been (and am still) working in the industry for 10+ years both as hardware systems/ research engineer, project lead, software developer and scrum master so far. I have a passion for playing the Bass.

Interesting name. How did you get it?

Schallbert is a portmanteau of the words Schall (a sound propagating through space) and Bert (a first name). Phonetically similar to Albert, I originally chose it as a player name: it is short and catchy, rare enough to work as a nickname without having to add numbers or glyphs, and, if you like, you can read into it a love of music and admiration for Albert Einstein.

Contact

via Github Discussions: schallbert.github.io
through my channel at PeerTube: CNC corner at MakerTube
via PlatformIO (µC-topics): Your account –> messages –> compose PIO Community
via Sorotec Forum (CNC-Topics): hobbyline.info
via CNCzone Forum (CNC-Themen): cnczone.nl -> Private Message
Directly (not preferred): See legal

Image courtesy of Likosdev ↩︎

AnywhereAmps - Alpha

Schallbert — Mon, 01 Jan 0001

Mobile Instrument Amplifier/Cabinet combo prototype based on Shelly13. AnywhereAmps Alpha has a low power, small footprint aproach. It is designed as a very portable full frequency range practice amplifier with a total weight of only 3.1kg. It features adjustable gain and a 9V DC supply for external effects pedals.

Project stats

Difficulty: medium 3/5
Cost: ~170€
Time: 10h

Impressions

Impressions of AnywhereAmps Alpha's prototyping

Can be played upright, especially suited for very small practice rooms.

9V voltage converter for floor pedals and accessories.

Minimalistic HMI: On/off switch, gain, 1/4'' jack, 9V out

12'' diameter, 2.6Gallons of volume

Battery on the back plate. Pull up to remove.

Preamp and poweramp are stacked on top of each other.

Amp sound samples

Bass samples

Your browser does not support the audio element.

Predominantly high notes

Your browser does not support the audio element.

Played really hard

Your browser does not support the audio element.

Open strings & flageolet notes

All tracks recorded with Zoom’s H4n using an Ibanez SR1500PD from 1992 with the neck pickups @100%, full volume, low string setting.

Dimensions

Raw drum shell: 12"x6". Total height is around 17cm, width 30cm, depth (without battery) at 32cm. The enclosed volume incl. amp and speakers is ~9l. I’m using a non-vented cabinet design.

Sound Hardware

Speakers: 4" Bass/Mid and two Piezo tweeters
Class D amplifier with 30W RMS power (bridged)
14.4V Li-Ion tool battery supply
All sensitive parts on the inside of the cabinet
Custom built preamp with 0...27dB adjustable gain

Simulation

The simulation shows that the volume of ~9l damped cabinet would be a perfect match for the selected 4" speaker. It will go down as low as 44Hz (F#1, one full note above e.g. a Bass’s lowest open string E1). There will be punchy lows at a precision which is still good. The downside is the low efficiency of just 78dB/W/m, so the expected maximum volume I’ll get from the power input will be rather low and the speaker will be driven into its excursion limits early.

Custom Preamp

The class-D amplifier I selected requires a line-level input, so I had to add a preamp to my design to get sufficient volume from the power stage. Even the cheapest Preamps like this one from Art Pro Audio would add considerable cost to the design. They would weigh a lot and consume quite some power, especially if they are tube preamps, and often have special requirements on input voltage (like AC only). Plus, their design wouldn’t blend into mine.

So I decided to design a low power, minimal parts gain stage myself. Click here to skip the nerdy Tech details

Voltage rails

As the preamp would amplify alternating voltage signals but I only have one battery voltage rail available, first I had to think about creating a virtual ground potential that would be the point of signal zero for the gain stage. Luckily, there are a lot of design guidelines about how to operate the amplifier with a single voltage source, so I had something to start with. I used 10kOhm resistors to clamp the inputs of the gain stage to the virtual ground.

Gain stage

AnywhereAmps Alpha should be a multi-purpose instrument combo, so that there should be a wide gain range available to bring everything from a passive guitar to a line level Keyboard to the same voltage levels for the power stage to deal with. That’s why I decided to go for a 0-27dB amplification range and selected resistors for the feedback loop to be 10kOhm and a 250kOhm logarithmic potentiometer.

Circuit simulation

I used Analog Device’s LTspice to simulate how this circuit might behave when given a typical Audio signal between 20Hz and 20kHz. From my studies I remembered that I might be most interested in a so-called Bode plot which would list both amplitude and phase over frequency in a double-logarithmic scale. Conveniently, LTspice offers such a function. As you can see, the preamp’s output - green - stays flat into the 10kHz range and the phase would only shift a little while the input line -blue - stays ~27dB below.

Op-Amp selection

Well. I tried this design on my breadboard with multi-purpose Rail-to-Rail Operational amplifier I had lying around and it worked… not well at all. When I connected a function generator at the input and measured what was supposed to be a sine wave at the output of the LM324, I didn’t get a sine wave at the output at all, but a half-sine with some steps and discharge-alike curves around 0, then followed by the negative half-sine… After some search I found out this is called “Crossover distortion” and is caused by the asymmetric output stage of this specific op amp type. I was able to correct this a bit when applying a higher load current on the output, but finally I wanted to choose an op amp which was better suited for my audio application without rail-to-rail capability and lower noise ratios. Thankfully, there are quite some pages on that topic like Cicfy Research’s Op-Amp shootout. Weighing in a reasonable piece price and the availability at my electronics distributor, I selected the NE5532P. The video below shows what I got from the updated op-amp selection.

Your browser does not support the video tag.

Preamping a Sine wave

Circuit design

I wanted my circuit board to use as little space as possible while keeping DC voltage supply panes like a frame on the outsides of the board. The board’s left half is dedicated to creating the virtual ground potential while the right half does the amplification part. The design is minimal, using only as much as four resistors and three capacitors.

For a technical deep dive, refer to my design considerations.

Sound quality

AnywhereAmps Alpha is absolutely quiet in idle, even with gain full up there’s only so much as a little hissing from the tweeters. No hum, no scratching when turning the gain knob. It works with any instrument due to the wide gain range using 0dB for line levels like Keyboards, preamped instruments, mobile phone, and up to +27dB for passive Bass or Guitar direct in. Maximum volume is fine although the low-lows “flatten down” too early, limiting usable volume range e.g. for organs and 5-string basses. For frequencies in the Bass range at high volume, it still comes down to Physics - the bigger the better. Find some sound samples up here.

Summary

For the cabinet, I ordered some raw custom size Tom-tom shells from Cube Personal Drums. Cutting and especially hand-routing a curved surface proved to be hard, and the fact that I don’t own a proper wood workshop and did all this work on the balcony of my flat didn’t make it easier. But after roughly 10h incl. soldering and electronics work, I now have more than a super-portable instrument combo.

It’s a piece of furniture.

I’m going to be playing it a lot. Plus, the good sound keeps me more motivated than ever to go the next step and boost reproduction of the low frequency range with another, larger prototype.

Pros:

I like the design.
Fits into standard Drum bags!
Lightweight, 3.1kg incl. Battery, 2.6kg without
Battery state of charge can be read at a glance
Retracted controls: Avoids inadvertent operation
More robust: All electronics and speaker chassis mounted inside
includes 9V, 1.6A power supply jack for effects, stomp pedals, etc.
Sounds good for any instrument as it plays almost the full audible frequency spectrum
Runs >20h on a battery charge (14.4V, 4Ah) as a practising amplifier and moderate volume
Runs >3h on a battery charge at maximum volume

Imperfections:

Too small for sub-bass reproduction.
No auto mute on switching the amp on or off -> Pop Noise
Of course, not enough volume to match a drumset or plugged band setup.
Looks weird with big battery packs on the back. Use of a “flat pack” would be nice

AnywhereAmps Alpha: DIY Replica

You want do build your own copy? Sure, use the step-by-step guide below and get going! AnywhereAmps Alpha DIY instructions

Your browser does not support the video tag.

Alpha live!

AnywhereAmps - Alpha 1.1

Schallbert — Mon, 01 Jan 0001

Mobile Instrument Amplifier/Cabinet combo prototype Version 1.1, evolving from AnywhereAmps Alpha.

Project stats

Difficulty: medium 3/5
Cost: ~200€ plus battery system
Time: 3h

Updates over Alpha 1.0:

Add power supervisor circuitry to protect both amplifier and battery pack from abnormal voltage levels / currents
Integrate on/off switch in potentiometer
Reduce complexity by using high efficiency full-range speaker instead of 3-speaker 2-way system
Add high efficiency switching buck converter to generate 9V effects voltage rail
Save construction time by using custom printed circuit board featuring preamp, step-down converter, and power supervisor
Save construction time by delegating most of the woodworks to a portal milling machine CNC¹
Eliminate switch-on pop noise by adding a RC-filter to the standby input of the power amp stage

Build variants

To get progress on sound for my prototyping, I built different variants this time. I changed speakers and evaluated the sound:

Alpha 1.0: 30W bridged amp, 4" bass and 2x piezo tweeters: HiFi-esque sound with punchy lows and clear highs, limited on the max volume side
Alpha 1.1 v1: 30W bridged amp, 4" PA driver, good volume and high presence, but underrepresented lows and highs
Alpha 1.1 v2: 30W bridged amp, 4" PA driver and passive membrane, fixing low-end

My own preamp and auxiliary functions printed circuit board

The AnywhereAmps 1.1 PCB features some extra functionality apart from the mere preamp plus lowpass and highpass for bi-amping. The details can be found in the following sections.

Power Supervisor

I added the power supervisor circuitry to not allow the battery to run completely empty. I also wanted to mitigate risks created by short circuits and other fault conditions which could harm the battery. The integration of this technology worked on first try (which I hadn’t expected). The only drawback is that when the battery gets discharged to below, say 10%, the amp would sometimes restart on power bursts as supply voltage would drop below the supervisor’s trigger voltage. As the battery voltage would go up again when load reduces, it will turn back on a second later.

This is clearly undesired behavior and will have to be handled with a refined prototype level. Another thing that makes this follow-up prototype necessary is the fact that I am not allowed to use the tool manufacturer’s batteries in my future designs as discussed here.

9V switching buck converter

To generate auxiliary 9V for devices like effects, loopers etc. I wanted to reduce noise by selecting a “quiet” switching regulator. The one I chose was designed for automotive applications and has the inductors included in its enclosure. The down sides are that the footprint is not suitable for hand soldering and the high price tag.

The converter would switch on when a device is plugged into the corresponding jack so that there’s no stand-by current. It operates quitely (but not completely without noise) and more than enough power for the applications I encountered so far - A looper, compressor, and tuner can be easily operated in parallel.

Woodworking

I bought a portal milling machine set to delegate the woodworking for AnywhereAmps to. I wanted to spend less time on cutting and sanding, and improve build quality as well as reapeatability. But changing the manual woodworks to more or less automated machining, I had to first move my plans into the digital world.

Computer Aided Design

I chose FreeCAD as design tool to create drawings. I documented the work with this tool in a couple of posts and continued to refine my workflow e.g. here. My first scetch to manufacture looked like this.

I was not completely happy with partitioning the sheet of wood yet. I found the legs too long and put some more hours in refining the design to use less space as the handling capacity of my portal milling machine is limited.

Computer Aided Manufacturing

When done, I exported the scetches as *.dxf files (Drawing Interchange File Format (DXF), introduced by AutoDesk in 1982. It eventually became industry standard for interchanging 2D drawings) and opened them in my CAM tool to define tool paths, set XY zero, and Z depth per pass.

G-Code

I ended up with G-Code files post-processed to match my machine with a forecast duration of 32 minutes per sheet. I was using just three tools: a 6mm endmill for the cutouts, a 2mm one for smaller holes and structures, and an engraving tool for chamfers and logo. I partitioned the work into three jobs:

job00_clamping to cut holes and countersinks into the sheet to allow attachment to the workbed
job01_nesting to do all cutouts and engravings on the ampcarrier, but leave taps so the parts don’t move
job02_finishing which cuts the logo on the (already coloured) top side and adds chamfers where needed

For finishing, I had to update CAD and CAM files so they would provide a mirrored (and turned by -90°) view of the sheet. This is to allow two-sided milling with the same XY-zero point and the exact same sides touching the fence for minimal tolerances.

Manual work remaining

I had to do a little manual work like sanding down taps, adding wax to get the top finish done, and cutting some holes for struts and stands that my CNC couldn’t do because they are on a plane orthogonal to the Z-axis.

I used the CNC to create adapters for my hand router so I could have it glide over the bent surface of the shell. This way, the holes that I cut in there for the speakers before could be properly chamfered.

I also had the CNC manufacture a shell holder in 3D-mode (with *.STL files). This way, the shell is kept in place when cutting holes for the speakers with my hand drill. By the way: 3D cuts take a lot of time as finishing uses small stepovers to get a smooth surface. I’d like to further improve shell handling; hand-cutting a >100mm hole does not allow good surface quality on the inside of the shell, and handling that huge drill is still a pain. Unfortunately, my CNC is not able to to this job for me.

Impressions

Impressions of AnywhereAmps Alpha V1.1

Battery attached and switched on

Front view, initial version of speaker protective grille

A view from below shows the passive radiator, extending the speaker's low-end

Preparing final assembly

A view inside: Ampcarrier and circuits

Allowing the wax to dry before final assembly

Computerized Numerical Control ↩︎

AnywhereAmps - Alpha 1.2

Schallbert — Mon, 01 Jan 0001

Alpha 1.2 v1: 2x15W bi-amped, 2x3.2" mid-high speakers, 8" subwoofer: very good low and mid, high highs a bit dull, weighs a lot
Alpha 1.2 v2: 2x15W bi-amped, 4" PA driver, 5.5" subwoofer, 2x piezo tweeters: Very balanced sound, build is pretty complex.

I’ll continue writing here once I selected the best variant.

CarSubAmp

Schallbert — Mon, 01 Jan 0001

project stats

Difficulty: beginner 1/5
Cost: ~50€ without battery pack
Time: ~2h

The CarSubAmp was my first prototype for a really portable Bass Combo (Speaker cabinet + amplifier) that doesn’t even need a wall socket.

Motivation

During Covid19 Pandemic, I enjoyed playing and excercising Bass outside in the Parks or on my balcony. Which is not a problem when wearing headphones and carrying a small preamp. But, there were occasions when I wanted to play along with, say, an Acoustic Guitar, or somebody would bring a Synth-App along with a MIDI Keyboard - and I was forced to use a small bluetooth speaker that neither had the output power nor the real-time feel.

Problem

My Bass amp is far from mobile. Weight: 18kg, size: huge, plus: It’s useless without a wall socket nearby.

Dead ends

Instead of a Class-D amp I used an old Car Radio’s aux in, but it seems to have expected “Line” level which my bass couldn’t supply so the output was rarely audible. Additionally, the idle current draw was >0.5A so it would drain the battery all too quick.
I tried the power supply with a powerbank (Power-Delivery) first, but the integrated step-up converter didn’t seem to handle full-volume current surges, so the supply voltage dropped, leading to audible distortion.

Solution

After some thinking, I came up with the idea of using a cordless multi tool’s Li-Ion battery that would be able to handle current surges easily, have a low weight, feature a battery level indicator and the typical protection circuits to avoid overload, short circuit, over temperature, exhaustive discharge etc. I wanted a Class-D power amplifier matched to the voltage range of this battery to be as energy efficient and as lightweight as possible.

The “CarSubAmp”

To check if the concept was viable, I took a used OEM car subwoofer assembly that normally goes into the spare wheel well, bought a 20W amplifier developer board and after soldering some components, adding a fuse and an 6.3mm input jack, my first prototype was ready to try.

Project summary

Very lightweight, <5kg
4Ohms Subwoofer assembly “ready to use”
vented design for better speaker efficiency
Sound is OK, but it’s clear that the assembly is not meant to be used as a full-frequency range speaker. Muffled.
Playing low notes (i.e. at or below g1), the system would become noticeably quieter
Developer board comes without preamp, thus maximum volume a bit low already
Under high load, vented design “Bassreflex” leads to audible hissing noise
Due to its small diameter, the speaker would hit maximum excursion limits when playing Hammer-ons on low strings
All components fixed with hot melt or tape, thus not very durable
Assembly is sealed with glue, no way of permanently adding a case or enclosure for battery, amp, cables, or input jacks

Altough this design had clear disadvantages, I actually enjoyed it a lot because for the first time, I could literally play anywhere and anytime I wanted. I even caught myself on the couch with this little amp although the “real” Amp was just one room away.

After some months carrying this thing around, 3 re-solderings of broken connections and glueing the battery back into place again and again, I decided not to stop with this prototype.

ClickEncoder

Schallbert — Mon, 01 Jan 0001

Project stats

Difficulty: medium 3/5
Cost: 0€
Time: ~6h

Abstract

Library for the Arduino Framework, written in c++.

It is lightweight because the source code itself is a few hundred lines only, and powerful, because of the improved algorithm uses less calculations while providing a better state interpretation than the original project this is forked from. It is officially released on PlatformIO IDE and hosted on Github. It contains examples both for Arduino and PlatformIO IDEs and provides unittests that can be run upon customization.

Motivation

I used the original library published by 0xPIT at first but that one caused some compiler warnings for me, plus the encoder readings were not flawless and sometimes there was jitter or bouncing I didn’t want to happen.

Solution

As the repo looked unmaintained, I decided to fix these issues by rewriting some code. For my project at that time, I also needed a “repeat” signal when a button is pressed continuously, so I added this, too. And finally, I separated the encoder algorithm from the button handling and provided separate classes so that, if needed, any combination of button and encoder could be used, each with configurable behavior.

Encoder Algorithm explained

API

ClickEncoder’s API is simple and easy to understand: service() runs the business logic and does the state interpretation. It has to be run regularly to check if the encoder has been moved since its last call, into which direction, how many steps, and has these values saved in an accumulator variable. getIncrement() returns the encoder step changes since its last call. getAccumulate() returns the sum of steps taken since startup. getButton() returns the current Button state.

void service();
int16_t getIncrement();
int16_t getAccumulate();
Button::eButtonStates getButton();

Encoder state

The encoder uses a Gray Code to encode the steps. getBitCode() converts this code transmitted via the Encoder’s hardware pins A and B to a 0...3 notch interpretation, numbers increasing when turning right, and decreasing when turning left. This is done efficiently without use of if() or other branching logic.

uint8_t Encoder::getBitCode()
{
 // GrayCode convert
 // !A && !B --> 0
 // !A && B --> 1
 // A && B --> 2
 // A && !B --> 3
 uint8_t currentEncoderRead = digitalRead(pinA);
 currentEncoderRead |= (currentEncoderRead << 1);
 
 // invert result's 0th bit if set
 currentEncoderRead ^= digitalRead(pinB);
 return currentEncoderRead;
}

I used a trick here, so you may have to look at the code snippet above several times. The state of PinA is written to bit1 of currentEncoderRead by me shifting it to the left with the << operation. But at the same time the state of the 0th bit is preserved by |=. If I now bind PinB simply by logical &=, then I would violate the Gray code, after which A&&B=2 should be and not 3.

So if A and B are both not set, the result remains 0, because 00^00=00=0. If only A is set, but B is not, the result is 11^00=11=3. If B is set, but A is not, the result is 00^01=01=1. If both are set, finally 11^01=10=2.

For more information about how logical binary operators work, please refer to my LED fireplace project

State interpretation

After the bit code is read, rawMovement variable is set that tells us how much the encoder has been turned between two runs of the service() routine. Here you can see the time criticality because there will be aliasing errors if the encoder changes notches in a time similarly short to the service interval.

void Encoder::handleEncoder()
{
 uint8_t encoderRead = getBitCode();
 // bit0 set = status changed, bit1 set = "overflow 3" where it goes 0->3 or 3->0
 uint8_t rawMovement = encoderRead - lastEncoderRead;
 lastEncoderRead = encoderRead;
 // This is the uint->int magic, converts raw to: -1 counterclockwise, 0 no turn, 1 clockwise
 int8_t signedMovement = ((rawMovement & 1) - (rawMovement & 2));

 encoderAccumulate += signedMovement;
 encoderAccumulate += handleAcceleration(signedMovement);
}

Here is another trick: signedMovement is calculated by converting the movement into a signed integer representing 0 as “not rotated, -1 as “rotated left” and 1 as “rotated right”. This exploits the integer overflow behavior, in combination with the interpretation of signedness.

Finally, the accumulation is performed and the acceleration, if configured, is added to the accumulator depending on how fast the encoder is rotated.

To program these two methods without branching took me several evenings of grubbling and practically half a pad of squared paper. Frankly, these few lines of code have brought me to the edge of my brain’s capacity. I am just not a “real” programmer 😅

The getAccumulate() method

It’s really simple, it just returns the internal accumulated value accounted to the encoder’s configuration about how many notches make a step.

int16_t Encoder::getAccumulate()
{
 return (encoderAccumulate / stepsPerNotch);
}

The getIncrement() method

Returns how much the encoder’s values have changed since its last call.

int16_t Encoder::getIncrement()
{
 int16_t accu = getAccumulate();
 int16_t encoderIncrements = accu - lastEncoderAccumulate;
 lastEncoderAccumulate = accu;
 return (encoderIncrements);
}

Button state calculation explained

The handleButton() method

All button states derive from the two basic states: pressed or not pressed. This is reflected in the logic as well:

void Button::handleButton()
{
 if (lastGetButtonCount < ENC_BUTTONINTERVAL)
 {
 return;
 }
 lastGetButtonCount = 0;

 if (digitalRead(pinBTN) == pinActiveState)
 {
 handleButtonPressed();
 }
 else
 {
 handleButtonReleased();
 }

 if (doubleClickTicks > 0)
 {
 --doubleClickTicks;
 }
}

The button can be configured to be read less often than the encoder. If this is the case, this method will be left without any action.

Button pressed:

handleButtonPressed() can either return Closed, Held, or LongPressRepeat depending on how the button is configured and how long it has been pressed already.

Button not pressed:

handleButtonReleased() can either return Clicked, Released, or Doubleclicked depending on button configuration and click count within a certain time.

Library configuration

The encoder/button detection behavior can be modified in the encoder.h file using the constants shown below. The values have been tweaked already and feel natural to me, at least. They should be modified if you choose to use a different service interval than 1ms because those values all take this interval as reference.

// ----------------------------------------------------------------------------
// Acceleration configuration (for 1ms calls to ::service())
//
constexpr uint8_t ENC_ACCEL_START = 150; // The smaller this value, the quicker you must turn to activate acceleration.
constexpr uint8_t ENC_ACCEL_SLOPE = 75; // the smaller this value, the stronger the acceleration will manipulate values.

// Button configuration (values for 1ms timer service calls)
//
constexpr uint8_t ENC_BUTTONINTERVAL = 20; // check button every x ms, also debouce time
constexpr uint16_t ENC_DOUBLECLICKTIME = 400; // second click within x ms
constexpr uint16_t ENC_LONGPRESSREPEATINTERVAL = 200; // reports repeating-held every x ms
constexpr uint16_t ENC_HOLDTIME = 1200; // report held button after x ms
// ----------------------------------------------------------------------------

Use it in your project!

The library is open source software, using the MIT license.
PlatformIO user? Easy, just search it in the ‘Libraries’ view, Tags are Arduino, Encoder, Schallbert. It’s then just one more click to add it to your solution
Alternatively, just clone or fork the repository and use the header/cpp file as you deem fit.
Contribute? Maybe you have a more efficient state calculation of the button? I’d be happy to review your pull request :)

CNC spoilboard

Schallbert — Mon, 01 Jan 0001

Project stats

Difficulty: medium 3/5
Cost: ~60€
Time: ~6h

Go to instructions

This is how to make a versatile spoilboard for your CNC application. The design uses hammer nut slots in different angles to enable very flexible clamping. It also includes a fence that makes preparing sheet material cuts easy: workpiece zero is at the same spot every time, and clamping against the fence might already suffice so that no load in Z-direction is needed. The fence has two cutouts: One to hold the tool-length sensor, and another one at XY-zero to mitigate tool collisions during rapid positioning.

Simple adjustment for your CNC

The dimensions I took are matching for a Basicline 0607 portal milling machine. But as I’m using a parametric CAD tool, all you need to do is to change dimensions to fit your machine, resolve some drawing quirks and you’re good to go with your own design.

The idea

As indicated above, the spoilboard has to meet the following criteria:

Mountable on CNC machine’s base frame (hammernuts / M5 fasteners)
Mount point close to XY-zero
T-slots for flexible mounting at 0°, 22.5°, 45°, 67.5°, 90°
Fence, quick-mount onto spoilboard to simplify clamping and improve repeatibility
Fence to hold tool-length sensor
Fence to provide a cutout at XY-zero
Shall be manufacturable on the CNC it’s designed for (multi-job if needed)

Design phase: CAD

I started with a dimensions table in Freecad and a couple of empty-sheet sketches.

Mounting to base frame

In the spreadsheet, I started by both entering the base frame’s and the CNC travel range’s dimensions, the drive-in nut’s, washer’s, and fastener’s parameters. I measured the exact positions of the base frame’s t-nuts where I wanted the spoilboard to mount onto and entered them in the spreadsheet.

When this was done, I opened the first sketch and drew the spoilboard’s rectangle, fully constraining its dimensions with the data from the spreadsheet. I added construction lines (i.e. lines that wouldn’t show in the final drawing, but help anchor constraints, draw lines, or mark symmetry within a drawing) where the base frame’s T-nut centers are positioned.

I added concentric circles for washer countersinks and through-holes in evenly distributed distances. They will later hold the fasteners and their washers. I didn’t decide on their final position on the Y-axis yet, as the spoilboard’s T-nuts would interfere with some of these mounting points.

The CNC job for this sketch will have to be done on the “top side” of the spoilboard.

T-nut slots and fence mounting points

Now it was time to add some data to the spreadsheet again: T-nut slot parameters, angles, border width like so:

First I had to add construction lines on the spoilboard to mark where the CNC’s travel range. Second, I had to reserve some room for the fence. Only then could I start to draw and position the T-slots which each had different angles and lengths based on the border I wanted to keep for stability reasons.

Every T-slot would get a symmetry construction line around which two rectangles reside to mark slot and rebate shoulders.

Then I added countersinks for bolt-in nuts that will later hold the fence on the spoilboard’s margin. In their center, I provide hole-throughs for the fasteners.

In a final step, I readjusted the base-frame mount sketche’s hole positions where they interfered with the T-nut slots.

The CNC job for this sketch will be executed on the “bottom side” of the spoilboard - the hammernuts will only work if they grip from below.

Fence

Just some more data needed to be added to the spreadsheet here: Ellipsoid dimensions for the XY-zero cutout, tool-length sensor, and the fence’s witdh on the Y-axis. Its dimensions on the X-axis were predefined by the machine’s travel area already.

Drawing the fence was easy because I could use the “link edge” tool to import positions of lines from other sketches. The fence is an own job with an own physical part on the CNC, and it is only being cut on the “top side”.

Planning phase: CAM

Now came the differcult part: Instead of three jobs as indicated above, I actually needed six. This is, because the CNC has less travel range is smaller than the spoilboard’s dimensions. I was further planning with stocks that already have the correct outline dimensions so I could fit them on the machine.

The first part of this job was creating sinks and holes for the lower end of the spoilboard mounts. I set the Zero at the very bottom-left of the machine (X0, Y-max) so I could be sure that the relative coordinates would fit.

For the upper end cut, topmost hole on the left was used as XY-reference zero created in the previous job. This way, I could clamp the stock more flexibly.

I repeated this approach for the other two jobs.

For the T-nut slot job, I had to mirror the sketch and turn my stock around the Y-axis as the spoilboard’s bottom needed to be machined here. This could be done in CAM as well, but I preferred to quickly jump back into CAD for this task.

For all these jobs, I used a 6mm 2-flute upcut wood bit with a cutter length of 21mm. As these were the first-ever steps I took with a CNC, I planned with very conservative parameters for this tool: S24000rpm, F2800mm/min, Z+=6mm, XY+=85%.

I even ran the first job “in the air” just to make sure I got everything set up right.

Execution

I scratched my head when I finally had the stock on machine bed: where could I clamp it? The sides were occupied by the portal, I have a wall at the rear of the machine… So would it be sufficient to only clamp it on the front? And as I was cutting all the way through, should I add another spoilboard below my spoilboard so that I won’t accidentally cut into the machine bed?

I decided that these objections are justified and waited until the weekend was over so I could buy some scrap wood and more double-sided tape.

This is how I clamped: Standard C-clamps at the bottom, double-sided tape at the top. Worked very well.

Before finally mounting the frame to the spoilboard, I leveled it with a “spiral” strategy. It was enough to cut Z=0.3mm. I used a 14mm face-mill cutter at a speed as low as S=5500rpm. This job took a long time, almost 30 minutes. The result was a surface as smooth as the original stock’s.

Assessment

Luckily, in the end it all worked out. Look how clean the cut is! Seems like MDF and sharp endmills are good friends. Note: MDF really needs a good dust collection as it will generate breathtaking amounts of dust when machined.

I’m quite happy with the overall job quality. Every single mount point to machine bed and fence fits OK and the tool-length sensor sits snugly in its cutout.

There are two things to criticize: The endmill I used is very loud and emits an uncomfortable shrieking noise that I even devoted a blog post to. Second, in my CAM tool, I didn’t care enough about milling direction. This way, the CAM decided to do one pass in conventional milling, another one in climb milling direction (it was a full slot as wide as the tool, so for the CAM it’s both climb milling, just the “side” of tool engagement changes). The difference is a very small but noticeable ridge on the fence’s edge.

With this spoilboard, I can process sheets as big as 620mm x 620mm now!

Well, one deficiency left is the tool-length sensor. It protrudes into the working area which is a problem when I want to machine large stocks and have tool changes planned: In this case, I’m using macros for measuring tool length which is why I don’t want to remove the sensor. As an iteration, I could rework the fence so it fully encloses the sensor.

Maybe.

Later.

Github Pages: Moving out

Schallbert — Mon, 01 Jan 0001

Project stats

Difficulty: medium 3/5
Cost: 5-50€/Month
Time: ~10h

Motivation

I decided to move my Website from Github-Pages to my own server. I have two reasons for that: First, I want the server to be situated in Germany (data protection) and second, I wanted a simple way of providing my page in two languages.

In addition, I might be able to realize additional wishes: Setup a miniature file server and add a Website for my small side-hustle. But those will be handled in a future post.

Overview

In this project, I’ll rent a virtual server, configure access via SSH and install a container manager called Docker.

In one of the containers, I want an instance of gitea to provide version controls for website, and have a CI/CD¹ pipeline setup that automatically integrates and deploys my site.

Not unlike Github, Gitea Actions allows me to deploy the site built with Jekyll via Workflow files on a caddy-Server instance.

caddy is not just a webserver, but also provides a reverse proxy which does Port mapping and address resolution. In addition, caddy manages my https certificates easily per Let’s Encrypt.

As caddy will run in an own docker container, I have to generate and configure virtual networks so the containers can “talk” to each other.

Let’s get going - step by step.

Configuring my server

The upcoming sections describe the individual steps of configuring a cloud-server for deployment of a static site, including version control and CI.

configure SSH for the server

Right after ordering the “Cloud-server”, I enabled SSH access and deactivated its password login. To connect from my machine, I check whether the ssh-agent is already up and running per eval ssh-agent.

Then, I create a new key pair via

ssh-keygen -t ed25519 -C "your_email@example.com" -f "~/.ssh/my-cloud-server"

This line creates a key pair with algorithm (-t) ed25519, taking a mail address as comment (-C) that is then saved with filename (-f) my-cloud-server.

The public key file my-cloud-server.pub is then uploaded on the web portal of my server using the “security” tab. The resulting fingerprint is then verified with:

ssh-keygen -lf <fingerprint_from_server> "~/.ssh/mein-cloud-server"

Finally, I modified the .ssh/config file so that, to connect, I just need to enter ssh server into my console instead of the server’s IP-Address - pretty Lazy 😀

#.ssh/config
Host server
 User root
 Hostname ip_address_of_server
 PreferredAuthentications publickey
 IdentityFile ~/.ssh/my-cloud-server

Now when I enter ssh server the first time, I get a prompt about a potential connection to a yet unknown host with fingerprint xyz and get asked whether I’d want that to happen. I compare this fingerprint to the server’s (use provider’s Website to verify). This way I can make sure that I’m trying to connect to the correct party. Once I give my yes, the fingerprint is added to the known_hosts file and future connections will be made directly and without warning.

Install Docker 🐳

My server runs a standard Ubuntu that supports apt commands out of the box. So installing Docker per Manual is really easy: Connect to the server per SSH, type apt install docker, done. I’ll be using docker-compose which also auto-installs. Compose interprets configuration files that define the target container and make it really easy to get them up and running.

Install caddy 🛒

First, I create a folder in my /opt directory via mkdir caddy2. To have caddy run in an own container, I change into that directory and create the following docker-compose.yml file:

#caddy2/docker-compose.yml

version: '3'
services:
 caddy:
 image: caddy:latest
 container_name: caddy
 ports:
 - "80:80"
 - "443:443"
 environment:
 - ACME_AGREE=true
 restart: unless-stopped
 volumes:
 - ./Caddyfile:/etc/caddy/Caddyfile
 - ./caddy_data:/data
 - ./caddy_config:/config
 - ./www:/www
 networks:
 - caddy-proxy

networks:
 caddy-proxy:
 external: true

This file tells docker to use latest caddy release. The container is named caddy and uses ports 80 (http) and 443 (https). Certificates are requested via ACME_AGREE and the container is able to access required files (Caddyfile, data, config) on my host machine via a volume. It will also be able to access files in /www folder that, in the future, will host my blog’s site data. The container is part of the caddy-proxy network that has been provided externally.

So, to be able to run this, I’ll first have to create that network:docker network create caddy-proxy. Then a quick check per docker network ls shows that the network has been added:

NETWORK ID NAME DRIVER SCOPE
<redacted> bridge bridge local
<redacted> caddy2_default bridge local
<redacted> caddy-proxy bridge local

Now I create a Caddyfile configuration with following content:

# caddy2/Caddyfile
blog.schallbert.de {
 root * /www/blog
 encode gzip
 file_server
}

Caddy is told that the content of /www/blog shall be provided as a (static) file server, supporting gzip-compression, available under blog.schallbert.de.

Create a test website 🧪

Before we start with the page design, we should be able to have the address resolved². I log into my DNS-manager and bind a static IP address to my subdomain:

TYPE NAME VALUE TTL
A blog.schallbert.de <redacted> 86400

My server shall be able to show something. So let’s create a minimalistic test page:

mkdir ~/caddy2/www/blog
cd ~/caddy2/www/blog
nano index.html

Hmm, Just a little something:

<!-- blog/index.html -->
<h1>ALL YOUR BASE ARE BELONG TO US</h1>

Save it, done.

This is how caddy’s root directory looks like now:

*caddy_config* *caddy_data* Caddyfile docker-compose.yml *www*

We’re ready now to create the container:

docker compose up -d

The Option -d means “detach”, so the container is started and detached from the console, unblocking it so we can reuse it for more prompts.

[+] Running 1/1
 ✔ Container caddy Started

This looks good. Now I navigate to the page:

Install Gitea 🍵

I need Gitea for two purposes: I want version control also for private repositories and I want to host my website with Jekyll with full CI/CD automation.

Like caddy, it starts with a folder:

cd /opt
mkdir gitea
cd gitea
nano docker-compose.yml

I compiled Gitea’s docker-compose.yml out of dfferent example files. Here it is:

# gitea/docker-compose.yml
version: "3"

networks:
 gitea:
 external: false
 caddy-proxy:
 external: true
 name: caddy-proxy

services:
 gitea:
 image: gitea/gitea:latest
 container_name: gitea
 restart: always
 environment:
 - USER_UID=1000
 - USER_GID=1000
 - SSH_DOMAIN=git.schallbert.de
 - GITEA_HOSTNAME=git.schallbert.de
 networks:
 - gitea
 - caddy-proxy
 volumes:
 - ./gitea:/data
 - ./log:/app/gitea/log
 - /etc/timezone:/etc/timezone:ro
 - /etc/localtime:/etc/localtime:ro
 ports:
 - "127.0.0.1:3000:3000"
 - "222:22"
 runner:
 image: gitea/act_runner:nightly
 environment:
 - CONFIG_FILE=/config.yml
 - GITEA_INSTANCE_URL=https://git.schallbert.de
 - GITEA_RUNNER_NAME=ichlaufe
 - GITEA_RUNNER_REGISTRATION_TOKEN=
 volumes:
 - ./runner/config.yml:/config.yml
 - ./runner/data:/data
 - /var/run/docker.sock:/var/run/docker.sock

This is what it does: gitea is member of docker networks gitea and caddy-proxy. The latter has been added so I can reach gitea from “the internet”. The container running the service is named gitea and is automatically restarted when down. User-ID is set to 1000 so it won’t interfere with admins or other special users. Per SSH and http, gitea can be reached under git.schallbert.de on ports 22 and 3000 - the latter is localhost only.

Gitea has access to the host’s time zone and local time, it saves its logs under /logand files under /data. Gitea gets an action runner that runs on the gitea instance. Its name is ichlaufe and the config is available under config.yml as a volume. The runner can access Docker daemon’s standard-socket to run actions, e.g. install ubuntu-latest in an own container to run Jekyll builds. I took the basic config for the runner from the official repository.

Gitea will only accept runners that are registered via GITEA_RUNNER_REGISTRATION_TOKEN. We’ll do this at a later point in time.

Now we start Gitea for the first time and look at the console print:

docker compose up
[+] Running 4/2
 ✔ Network gitea_default Created 0.0s 
 ✔ Network gitea_gitea Created 0.1s 
 ✔ Container gitea Created 0.1s 
 ✔ Container gitea-runner-1 Created 0.0s 
Attaching to gitea, gitea-runner-1
gitea | Server listening on :: port 22.
gitea | Server listening on 0.0.0.0 port 22.
gitea | 2023/11/18 17:19:52 cmd/web.go:242:runWeb() [I] Starting Gitea on PID: 15
gitea | 2023/11/18 17:19:52 cmd/web.go:111:showWebStartupMessage() [I] Gitea version: 1.21.0 built with GNU Make 4.4.1, go1.21.4 : bindata, timetzdata, sqlite, sqlite_unlock_notify
gitea | 2023/11/18 17:19:52 cmd/web.go:112:showWebStartupMessage() [I] * RunMode: prod
gitea | 2023/11/18 17:19:52 cmd/web.go:113:showWebStartupMessage() [I] * AppPath: /usr/local/bin/gitea
gitea | 2023/11/18 17:19:52 cmd/web.go:114:showWebStartupMessage() [I] * WorkPath: /data/gitea
gitea | 2023/11/18 17:19:52 cmd/web.go:115:showWebStartupMessage() [I] * CustomPath: /data/gitea
gitea | 2023/11/18 17:19:52 cmd/web.go:116:showWebStartupMessage() [I] * ConfigFile: /data/gitea/conf/app.ini
gitea | 2023/11/18 17:19:52 cmd/web.go:117:showWebStartupMessage() [I] Prepare to run web server

Configure web access

Now I have another subdomain added to my DNS manager:

TYPE NAME VALUE TTL
A git.schallbert.de <redacted> 86400

I also have to modify Caddyfile so that web requests are forwarded accordingly (reverse proxy). To do this, I navigate back to the caddy2 folder, type nano Caddyfile and add the following section:

# caddy2/Caddyfile
git.schallbert.de {
 reverse_proxy * http://gitea:3000
}

Now requests to git.schallbert.de are forwarded to the (local) port 3000 of the Gitea-container that I opened in the docker compose file before.

Configuring Gitea per web page

When I use my browser to navigate to the above page, I get a login screen - identical to this demo.

Now I create a nice password, click Register and then configure Gitea using my browser. I only do a rough config because I’ll have to modify it again later to properly add the runner.

When done, I create a new SSH key pair as described in the section Configure SSH above with filename -f ~/.ssh/gitea. It is uploaded and verified under Settings->SSH/GPG Keys.

Now I can create another entry on my local machine’s .ssh/config file:

# .ssh/config
Host gitea
 User schallbert
 Hostname git.schallbert.de
 PreferredAuthentications publickey
 Identityfile ~/.ssh/schallbert_gitea

Let’s check if this works:

ssh gitea
Host 'git.schallbert.de' is known and matches the ED25519 host key.
debug1: Found key in /home/schallbert/.ssh/known_hosts

Perfect.

Complete Gitea config via `app.ini`

As the configuration is not complete, I shut down the container again:

docker compose down

and then navigate to /gitea/gitea/conf and modify the app.ini just a little:

# conf/app.ini
APP_NAME = Gitea: Git with a cup of tea
RUN_MODE = prod
RUN_USER = git
WORK_PATH = /data/gitea

[repository]
ROOT = /data/git/repositories
DEFAULT_BRANCH = main

[repository.local]
LOCAL_COPY_PATH = /data/gitea/tmp/local-repo

[repository.upload]
TEMP_PATH = /data/gitea/uploads

[server]
APP_DATA_PATH = /data/gitea
DOMAIN = git.schallbert.de
SSH_DOMAIN = git.schallbert.de
HTTP_PORT = 3000
ROOT_URL = https://git.schallbert.de/
DISABLE_SSH = false
SSH_PORT = 222
SSH_LISTEN_PORT = 22
LFS_START_SERVER = true
LFS_JWT_SECRET = <redacted>
OFFLINE_MODE = false

[database]
PATH = /data/gitea/gitea.db
DB_TYPE = sqlite3
HOST = localhost:3306
NAME = gitea
USER = root
PASSWD =
LOG_SQL = false
SCHEMA =
SSL_MODE = disable

[indexer]
ISSUE_INDEXER_PATH = /data/gitea/indexers/issues.bleve

[session]
PROVIDER_CONFIG = /data/gitea/sessions
PROVIDER = file

[picture]
AVATAR_UPLOAD_PATH = /data/gitea/avatars
REPOSITORY_AVATAR_UPLOAD_PATH = /data/gitea/repo-avatars

[attachment]
PATH = /data/gitea/attachments

[log]
MODE = file
LEVEL = info
ROOT_PATH = /data/gitea/log

[security]
INSTALL_LOCK = true
SECRET_KEY = <redacted>
REVERSE_PROXY_LIMIT = 1
REVERSE_PROXY_TRUSTED_PROXIES = *
INTERNAL_TOKEN = <redacted>
PASSWORD_HASH_ALGO = pbkdf2

[service]
DISABLE_REGISTRATION = true
REQUIRE_SIGNIN_VIEW = false
REGISTER_EMAIL_CONFIRM = false
ENABLE_NOTIFY_MAIL = false
ALLOW_ONLY_EXTERNAL_REGISTRATION = false
ENABLE_CAPTCHA = false
DEFAULT_KEEP_EMAIL_PRIVATE = false
DEFAULT_ALLOW_CREATE_ORGANIZATION = false
DEFAULT_ENABLE_TIMETRACKING = true
NO_REPLY_ADDRESS = noreply.localhost

[lfs]
PATH = /data/git/lfs

[mailer]
ENABLED = false

[openid]
ENABLE_OPENID_SIGNIN = false
ENABLE_OPENID_SIGNUP = false

[cron.update_checker]
ENABLED = false

[repository.pull-request]
DEFAULT_MERGE_STYLE = merge

[repository.signing]
DEFAULT_TRUST_MODEL = committer

[oauth2]
JWT_SECRET = <redacted>

[actions]
ENABLED = true

Here it is important to make sure that ROOT_URL and port configuration match the one specified in docker-compose.yml. I configure its database (sqlite3) and make it available to Gitea. Plus, I deactivate Giteas ability to register new users DISABLE_REGISTRATION = true and the option to sign up/in via openID ENABLE_OPENID_SIGNIN = false ENABLE_OPENID_SIGNUP = false. Finally, I enable Gitea Actions ENABLED = true and save the file.

Now I can restart the container:

docker compose up
[+] Running 4/4
 ✔ Network gitea_default Created 0.1s 
 ✔ Network gitea_gitea Created 0.0s 
 ✔ Container gitea Started 0.0s 
 x Container gitea-runner-1 Error 0.1s

Very well, Gitea is up and running. In a last step, I configure the Action Runner.

Setup Action Runner 🏃

I use the official documentation provided by Gitea for inspiration.

I login to Giteas Web surface and see that activating Actions in the app.ini added another menu: Settings->Actions. Here I click Create new Runner und copy its REGISTRATION TOKEN.

Then, I re-open Gitea’s docker-compose.yml and add the following lines as recommended in the doc page:

# gitea/docker-compose.yml
# [...]
runner:
 image: gitea/act_runner:nightly
 environment:
 - CONFIG_FILE=/config.yml
 - GITEA_INSTANCE_URL=https://git.schallbert.de
 - GITEA_RUNNER_NAME=ichlaufe
 - GITEA_RUNNER_REGISTRATION_TOKEN= <redacted>

I restart the container with docker compose down && docker compose up -d and get a “started” message for the runner in the logs:

✔ Network gitea_default Created 0.1s 
✔ Network gitea_gitea Created 0.0s 
✔ Container gitea Started 0.0s 
✔ Container gitea-runner-1 Started 0.1s

Great, let’s have a look at the web page:

Now we have to know if the runner can do something for us.

Runner functional test

Once again, I have a look at the Runner’s documentation. To check, I create a new Repository on the web surface, name it runner-test, and add a file to /.gitea/workflows/01-test.yml containing the recommended runner check.

Commit, push, and - nothing.

Oh, the workflow starts on: [push]. This cannot work when it has just been uploaded itself.

Well, let’s touch another random file in the repository. Commit, push, result:

Fantastic! 🥳

Interim conclusion

In this article, I pretended that the road to a working server setup was straight and without obstacles. Of course, that is not entirely true. It was really simple to run images in Docker, but to have them communicate turned out less straight-forward. I also had quite some problems with Caddyfile as it is hard to debug and I couldn’t see what I was doing wrong.

For instance, I was able to ping the website on my host but couldn’t reach it from the outside. So I knew it is the caddy config to modify.

Root cause was missing the Caddyfile in the Docker volume list and forgetting a leading / in the target volume folder.

On the other hand, configuring Gitea was much easier than I thought. Even complex process of registering the runner and its routing through Docker worked at first try.

Now I have a quick and lightweight server that will be able to host my blog with ease.

Next steps - creating a build process for my site - is tried in this post, but then replaced with native Jekyll over here.

Ultimately, I want to get back to creating content. And not spend any more time than necessary on all the other stuff.

CI/CD “Continuous Integration / Continuous deployment” cares for automatic build, verification, and deployment of my web page. ↩︎
DNS = Domain Name System, a distributed hierarchical system for name resolution of websites. It exists because it’s much less straight forward to enter an IPV6 address like 2a02:ec80:300:ed1a:0:0:0:1 into your web browser than en.wikipedia.org. ↩︎

GoogleTest Demo

Schallbert — Mon, 01 Jan 0001

Project stats

Difficulty: medium 3/5
Cost: 0€
Time: ~2h

Motivation

It is nice to have a powerful unit test framework at hand when developing more complex projects, possibly saving a major amount of time debugging (especially on-Target debugging in the embedded world can be really time-consuming), improving code quality and possibly boosting insights to APIs and architecture for others.

The embedded world

Traditionally, microcontrollers were limited on the memory side. In the past, they neither had much RAM nor PROGMEM so the code I wrote was mostly in lean C, and configuration took place in lists of #define that wouldn’t weigh much because the preprocessor would do the work; not the µC at runtime. My unit tests were alyways bound to the hardware they were running on - take Unity as a well-known example - but now I wanted to try something new: Write Object-oriented style code, and get the business logic tested independently of the hardware. I selected the GoogleTest framework and I wanted a seamless integration into my workflow with VScode and its extension PlatformIO.

Until recently, the only way to do this was to run GoogleTest in parallel using the GNU Compiler Collection and have it controlled by PlatformIO. Recently, however, GoogleTest can be used directly as a PlatformIO Library. Therefore, I’ll cover both options here.

Using GoogleTest directly in PlatformIO

The installation is particularly easy here:

Clone my Github repository PlatformIO-gTestgMock.
Open a console window in the repository folder and switch to the corresponding branch using git checkout googletest-native.
Follow the instructions in the repository’s README file and test the setup.

This is what it looks like when the tests have been successfully run:

Using GoogleTest with GCC

You can install the required software yourself (even on a Windows¹ computer) to get started with GoogleTest.

Download and install MinGW

See this thread in the PlatformIO forum if you have any questions about the following steps.

Unix users: Skip the steps up to this point, as you probably only need the libpthreadgc libraries. Check for the presence of all libraries (but omit the mingw32- prefix) and install any missing ones. Make sure the PATH variable is set correctly.
Windows users:

Download MinGW, e.g., from Sourceforge
Install by running mingw-get-setup.exe. If you like graphical user interfaces, leave the corresponding box checked. I do not recommend changing the installation directory. If you do, you’ll need to remember the new path.

Configure MinGW

For GoogleTest to run, you’ll require additional packages to be installed:
- mingw32-gcc-g++ (of course!)
- mingw32-libmingwex
- mingw32-libmingwex-dll
- mingw32-libmingwex-dev
- mingw32-libpthreadgc-dll
- mingw32-libpthreadgc-dev
To install these,
- If you use the GUI: search libraries under “MinGW Standard Libraries”, add/activate/install them one by one.
- Command Line Interpreter users: type mingw-get install and then add the package name. Easy. If you get an error message saying that some of those do already exist, that’s good for you!
Now you need to add MinGW’s \bin folder to your system path variables so PlatformIO can find them later. If you didn’t hand-modify it, it should be C:\MinGW\bin.
- GUI fans: follow this guide
- CLI: type set PATH=%PATH%;C:\MinGW\bin, check if it worked with echo %PATH:;=&echo.%.

Congratulations! You have installed MinGW, set its environment variables to be found more easily by third party applications, and configured it to work with GoogleTest.

Configure PlatformIO

After setting the PATH variable, you’ll have to restart VScode.
To make PlatformIO correctly link to the compiler you just install, make it use the native environment where your compiler sits. Open a PlatformIO terminal within one of your projects and type pio platform install native
Two choices:

Follow through this tutorial and use my example project as described below. You can safely skip the upcoming section as that project comes pre-configured and is open for your modifications as you have basic hardware abstraction interfaces at the ready right from the start.
Take your own project and modify its Platformio.ini so that GoogleTest can be run.

This what you should add to your Platformio.ini:

[env:desktop]
platform = native
build_flags = 
	-std=gnu++11 # use installed GNU C++11 compiler.
	-pthread # found in gtest documentation
lib_deps = 
	googletest # Will automatically load latest googletest lib
lib_ignore = 
 src # most main.cpp's directly access the hardware. 
 # Can be removed if your project is different.
 #
 # PLUS: Any files that contain bare-metal hardware commands 
 # (e.g. digitalWrite), g++ won't have the headers or specialized compiler knowledge!
lib_compat_mode = off # Must-have for external stuff like gtest!

For the environment desktop (= your computer without target), it uses the native compiler which is GCC/G++ that you installed and set the PATH for. GoogleTest uses C++11 standard and threading which you have to add to the build flags. Of course, the build depends on the googletest library and you want to ignore all files/folders that contain source or header files using hardware-related commands or headers. Library compatibility mode has to be switched off for the Library Dependency Finder to include googletest.

(Optional) In case you like using GUI buttons better than typing a line into your console to start the tests, create a custom configuration.

Running Tests

For a quicker start, you can clone my example repository (on branch main). It contains all the necessary elements to test whether gTest and gMock are working correctly. This allows you to immediately verify whether the installation was successful.

Unzip the zip file or use SSH to clone the repository.
Open the PlatformIO extension home page, click “Open Project,” and select the folder.
To run the unit tests, open a PlatformIO terminal and enter pio test -vvv -e desktop -f test_desktop (-vvv = verbose -e = environment, -f = filter). The parameters are required because the tests can only be run on the desktop environment, not on the microcontroller.
After a while, the message shown below should appear.

Great! Unit tests within PlatformIO have been built and executed!

Tips and Tricks

Note that your PlatformIO_gTestgMock/.pio/build/desktop folder now contains an executable called program.exe. If you run this from the CLI of your choice, you should see the same result as in the above image. You might want to choose this option more frequently once you have more tests that not only fail but crash the program on execution, and to debug tests (and not the user code), as PlatformIO’s CLI tends to hang or miss out the reason for why the test crashed. You might also want to have the stack trace printed in these cases although this would really go far beyond scope of this exercise.

From here,

you might want to continue reading below for a deeper understanding of the repository contents and how GoogleTest works
you could go practice writing tests by adding more cases, e.g. to verify if the LOW state is looped through correctly
You could also enhance the interface of these groundworks, e.g. by adding the analogRead() function to the hardware abstraction layer
You could write your project logic on top of this, cleanly cut out the hardware dependencies, and enjoy the ability for easy porting of your project to different hardware devices and manufacturers
Or do this other brilliant stuff that you were up to before you were diverted by this tutorial.

Repository contents explained

The following sections cover most of the files in the repository and explain what they do in detail. The implemented business logic is minimal and should be seen as example code - to create the least confusion possible when dealing with this complex matter. For a real-life project that uses all of the patterns illustrated on this page, please refer to my Tonuino page.

Hardware abstraction layer interface

Let’s have a look at the bespoke interface and how it’s structured.

// hal_if.h
namespace hardwareAbstraction {

// Hardware Abstraction Layer Interface class
class Hal_IF
{
public:
 virtual ~Hal_IF() = default;

 virtual bool digitalRead(uint8_t pinId) const = 0;
 virtual void digitalWrite(uint8_t pinId, bool value) const = 0;
};

} // hardwareAbstraction

The Hal_IF class has two methods which are both virtual. This means that a client class using this interface can call the method, and is thus “auto-forwarded” to the concrete implementation of the method. The = 0; makes these methods pure virtual, so overriding them with a derived class is not optional but a must. This is what we want here, because if this method wasn’t overridden, it wouldn’t have any behavior. Note that the destructor of an interface class almost always should be virtual, too.

LoopInOut

The logic under test. Note that it does not directly access the input/output hardware of the µC, but the m_hal variable which is an implementation of the Hal_IF interface at runtime. It is passed into the class by reference (marked by the & at the type declaration) upon object construction².

// loopInToOut.h

class LoopInToOut
{
public:
 LoopInToOut(hardwareAbstraction::Hal_IF& hal): m_hal(hal){};
 ~LoopInToOut() = default;

 bool loopThrough(uint8_t inPin, uint8_t outPin);

private:
 hardwareAbstraction::Hal_IF& m_hal;
};

This is enabling the real magic of gMock: as the client code works with an interface, I can hand over the “real” implementation (of the hardware access) at chip runtime, but for the test I can hand over a “mock” implementation which I have control over at the time the test is running.

// loopInToOut.cpp

bool LoopInToOut::loopThrough(uint8_t inPin, uint8_t outPin)
{
 bool input = m_hal.digitalRead(inPin);

 m_hal.digitalWrite(outPin, input);
 return input;
}

loopThrough() again is really simple. It reads the input pin’s state and maps it to the output pin. It returns the input pin state to the caller.

Mock Hal

GoogleTest uses macros to define behavior. The mock class is also deriving from the interface and just provides declarations of the interface method it overrides. There’s no behavior involved just yet.

// hal_mock.h

class Hal_mock : public hardwareAbstraction::Hal_IF
{
public:
 MOCK_METHOD(bool, digitalRead, (uint8_t pinId), (const, override));
 MOCK_METHOD(void, digitalWrite, (uint8_t pinId, bool value), (const, override));
};

The test fixture

The test fixture contains common data and behavior for all test cases within a test suite. Test suite name = class name, and this exact name has to be reused in the tests that should belong to this suite.

// unittest_loopInToOut.cpp
class loopInToOut_Test : public ::testing::Test
{
protected:
 virtual void SetUp()
 {
 m_loopInToOut = new LoopInToOut(hal_mock);
 }

 virtual void TearDown()
 {
 delete m_loopInToOut;
 }

 NiceMock<Hal_mock> hal_mock{};
 LoopInToOut* m_loopInToOut{nullptr};
};

Thus, every test has some data prepared it can use: the class under test is set up as a pointer with a mocked implementation of its dependency.

The actual test

The test is defined by the TEST_F macro meaning “test with fixture” that expects test suite and test names. The ON_CALL macro is a command to gMock providing behavior to the mocked class how to behave when digitalRead(_) is called with any input (underscore = any): It shall return true by default.

// unittest_loopInToOut.cpp

TEST_F(loopInToOut_Test, loop_inputHigh_writesCorrectOutputHigh)
{
 ON_CALL(hal_mock, digitalRead(_)).WillByDefault(Return(true));

 EXPECT_CALL(hal_mock, digitalWrite(2, true));

 m_loopInToOut->loopThrough(1, 2);
}

Then there’s an expectation expressed on the mocked class, stating that digitalWrite is called with input values of (2, true) which are also checked for correctness.

The last statement actually calls the implementation so that the expecations can be verified.

Hardware behavior implementation

As main.cpp accesses the hardware, there has to be some kind of interface implementation for this.

// hal.h
class Hal : public Hal_IF
{
public:
 Hal() = default;
 ~Hal() = default;

 bool digitalRead(uint8_t pinId) const override;
 void digitalWrite(uint8_t pinId, bool value) const override;
};

The header states that Hal derives from Hal_IF and overrides both of its methods.

// hal.cpp
#include "Arduino.h"

#include "hal.h"

namespace hardwareAbstraction{

bool Hal::digitalRead(uint8_t pinId) const
{
 return digitalRead(pinId);
}

void Hal::digitalWrite(uint8_t pinId, bool value) const
{
 digitalWrite(pinId, value);
}

Note that only the source file actually includes the Arduino.h to access the hardware. So the dependency to hardware functions is very limited and buried in interface implementations that can be selected by the program’s structure. While main.cpp will want to create an object of the above implementation, the test will want to instead hand over a mock.

Gtest key concepts

Google’s official github page on GoogleTest will help you a lot, not only for getting started³. The gmock Cookbook is an excellent guide that I have been using over the months as my tests became more complex hard to write, and the gmock Cheat Sheet will help you when you lost track of how to write a test for this and that special problem.

Dependency Injection

Dependency Injection is the basic mechanics you need to understand to make gMock work. Let me draw an image for you to make it more apparent:

The important concept is, that the interface is merely a placeholder for the implementation of the client classes’ dependency. At runtime, e.g. when the host creates the client object, it will handover (i.e. inject) the implementation of the dependency into the client which then can call any of its methods, as their signatures are known at compile time due to the interface that the client is using.

The mock now merely is an implementation of the interface specialized for testing.
The production implementation is used in all other contexts, it features the “real” behavior of that class.

Hardware Abstraction Layer

Yep, well, this might be one of the tricky bits when working with embedded software. At some point, it boils down to outside world interaction, be it with General Purpose Input Output (GPIO) pins, accessing one of the myriad of hardware bus systems like I2C, UART, SPI, CAN to control external hardware, or simply to use system internals like the EEPROM - that’s where the trouble begins.

Gtest runs on GCC/G++ compiler which just doesn’t know anything what the specialized compiler of that particular chip knows. It wouldn’t even understand the header files. Now, there are two options around this issue:

Write fake headers for each µC-specific thing that can be used by GCC/G++
Cut! At! The! Interface! to any µC-specific thing using an (Hardware) Abstraction Layer so your production environment uses the chip hardware as it normally does, but the test environment uses mocks instead.

There are quite some advantages for the second approach:

It’s clean. It will force you to not cross abstraction layers which should not be crossed.
It’s easy. You can test your business logic without even programming your µC! Just mock the inputs and intercept the outputs, write some tests and run them in the terminal.
It’s portable. Exchange the implementation of your abstraction layer, and you’re almost ready to execute on a different controller family or even on another manufacturer’s chips.

Drawbacks

Hmm, there must be some shadow somewhere. Right you are. In my opinion, these are the main disadvantages when getting everything ready to be tested without hardware attached:

Memory. Both ROM and RAM will be cluttered more easily when you have a massive amount of classes and vtables flying around.
Ownership. Injecting dependencies separates ownership from useage of an object, and you need to make up your mind how to organize both.
3rd party libaries. You’ll have to create an interface header, an adapter implementation, and a mock for each library that uses “chip-internals”.
Complexity. As your objects are injected via an interface, you’ll have to use call-by-reference or hand over by pointer 😨 now. The former is additionally limited because it will only work for constructor injection, not for any other kind (method injection, setter injection, etc.)

Some more tips

Keep everything “behind” your abstraction layer as simple as possible: You can only test and debug this stuff on target. Example: each method only executes one command (as in my example code).
Create a loader class or a Dependency Injection Container (Nicolas Croad) that holds all objects to be injected and can spit them out on request.
Make up your mind about the drawbacks mentioned above and how you create your low-level Architecture to accomodate for this.
Don’t hesitate to create more than one level of interfaces. It might be a good idea to use this pattern not only for the direct hardware interaction, but also in between software modules, which then can be tested in a higher granularity so your overall progress might even be quicker versus one big ball of classes you can just control via mocks in the Hardware Abstraction Layer.

Tough Stuff!

You earn a break now. This was much information to deal with. Take your time. Contact me on this topic’s discussions page if you feel something is missing, incorrect, to be improved, or just exactly what you needed right now.

Schallbert

Installation under Linux is actually more easy, as gcc and g++ come already pre-installed. So even if they do not feature the correct packages yet to support GoogleTest, their error message will be of much better help than the ones you get on Windows when some libraries are missing… 🙃 ↩︎
This is also called Constructor Injection, a subtype of Dependency Injection which is a means to achieve Inversion Of Control. I have prepared some links for you to continue reading, it is a rich topic that whole books have been written about.` ↩︎
By The way: the googletest site is also static like mine and it uses Jekyll as well 👋 …do you like it? I’d like to see your opinion on my discussions page ↩︎

Legal / Imprint

Schallbert — Mon, 01 Jan 0001

Legal notice

Address

Contact

Editors

Source

e-recht24.de

Dispute settlement

I’m neither willing nor obliged to participate in a dispute settlement hearing.

Liability

Did I make a mistake or forget anything?

Errare humanum est. I’m trying to run this website according to the latest legal requirements. Should anything on here be non-compliant to these regulations, I humbly request for an informal note. I’ll promptly tackle valid complaints to remove deficiencies.

As a service provider, I am liable for my own content on these pages in accordance with Section 7(1) of the German Telemedia Act (TMG) and general legislation. However, pursuant to Sections 8 to 10 of the TMG, I am not obliged, as a service provider, to monitor third-party information that is transmitted or stored, or to investigate circumstances that indicate unlawful activity.

Obligations to remove or block the use of information in accordance with general laws remain unaffected by this. However, liability in this regard is only possible from the time I become aware of a specific legal infringement. Should I become aware of any such infringements, I will, of course, remove the content in question immediately.

Disclaimer

As a service provider, I’m committed to meet German law for the contents I author. I am not obliged to monitor transmitted or saved contents of third parties or to research for potentially illicit behavior. My Website contains links / references to third party websites over which I have no control. That’s why I cannot take any responsibility for their content. I have checked third party links for legal compliance at the time of linkage and couldn’t find any illicit behavior.

Copyright

The contents I prepare for this website are subject to German copyright. Copies, modifications, and usage beyond this require my written approval. Downloads and copies of this Website are allowed for non-commercial, private use.

I’m trying hard to mark third party content and to obtain written consent prior to their usage. Should you still spot a copyright violation, please drop me an informal note. I’ll be happy to make my website even more transparent!

MobFobAmp

Schallbert — Mon, 01 Jan 0001

A second prototype of my mobile Instrument Amplifier/Cabinet combo with foldable design. 28l cabinet with 17cm woofer. 15W amplifier power. 7.3kg incl. battery. Especially suitable for Bass Guitar, Acoustic guitar, Organs.

Project stats

Difficulty: challenging 4/5
Cost: ~140€ without battery pack
Time: ~15h

Impressions

Some images of my fold-flat bass combo 'MobFobAmp'.

Consumes very little space when folded

Easy to carry, weight is less than 8kg.

28l of volume when unfolded.

Height of 20 inches, can be used as a stool.

6.5 inch low-end driver, 4Ah battery pack for hours of battery life.

A peek inside

Your browser does not support the audio element.

MobFobAmp Sound sample

Your browser does not support the audio element.

A/B/A comparison Sealed vs. Vented

Your browser does not support the audio element.

MobFobAmp and Piano, Sealed, Preamped, at maximum volume

All of these tracks have been recorded with my Smartphone, so no high-end gear used. Still, I think they represent the actual sound of MobFobAmp quite well.¹

“MobFoBamp”: Motivation

To overcome many of the constraints of my previous prototype, I decided I’d need a bigger speaker, and a more voluminous enclosure to yield a better response on lower frequencies. Unfortunately, a bigger enclosure would considerably reduce mobility, so maybe I could get rid of all the air inside the box for carrying…?

The Idea: Folding box

A folding box ^[1]

Following the concept of a folding box like shown, I decided my combo should be collapsible for easy carrying. I chose a sealed design for starters so I wouldn’t have to care about calculating vents, resonance frequencies and group delays.

I had to think hard how to integrate speakers and electronics while keeping the ability to fully fold the cabinet.

Dimensions

I selected the cabinet’s width in a way that it would fit snugly into my bass gig-bag if needed. All other dimensions fell into place by itself - height had to be at least twice the width so top and bottom lids could be accomodated when folded (but not too high to keep stiffness as high as possible. Depth had to be less than (0.5* width) - (materialStrength + hingeThickness), otherwise folding wouldn’t work as the sides would get in their way.

Sound Hardware

For the power Amplifier I chose to stick with the module I already bought earlier. I chose a 6.5inch Hi-Fi woofer (50W, 4Ohm) with low resonance frequency to match my sealed cabinet, which I figured would have a volume of ~28l.

Simulation

Then I ran some simulations to see whether cabinet design and speaker would be a good match.

Estimated Qtc would be at 0.63 instead of the recommended 0.71 (optimal step response), so the cabinet would be somewhat overdamped and sound “tighter” but loose volume at the low end compared to an optimal setting. As you can see, building a cabinet always is a compromise between sound, power, size, weight, frequency response, and volume level that can be obtained from that setting.

Summary

When I finished building that cabinet after some 15h, I was quite happy to see the cabinet is airtight and not flimsy at all. Unfolding would take me below 60 seconds, folding it back to its original state even less than 30 seconds. As maximum level still was a bit low, I added a compressor stomp pedal to the setup which had a gain knob I could use as a preamp.

The sound of this thing is amazing. Tight Bass, warm mids and subtle, but brilliant highs fitting the characterisitc sound of the bass guitar very well. Maximum sound level is limited, but it’s always enough to keep up with an acoustic guitar. I’m using it as a practising amp all the time.

Sound quality

For sure, the most important aspect of an instrument combo is how it sounds like. Due to the foldable design, it can be operated in two different setups.

Sealed: Tight and well-balanced sound, good low-end punch. Without preamp, maximum level OK to play along with an Acoustic Guitar
Vented: Bottom Lid opened ~3cm, latch suspending cabinet against bottom panel. Much louder sound with pronounced low-mids, less balanced and a bit boomy, maximum level on par with Piano played f.

I’m really happy with the sound quality MobFobAmp provides. The option to play “open” adds flexibility and makes the sound really stand out (with the drawbacks mentioned above).

I also have performed some measurements to see how close the design gets to an ideal frequency response, sending some White Noise to MobFobAmp and then recording the sound output with a microphone. For comparison, I displayed a spectrum analysis in the frequency domain. A perfectly neutral sound output would hit the orange line at 0dB for the whole spectrum.

You can see, that the -3dB limits or useable frequency range is from 30Hz to ~1.5kHz.

Add-Ons

9V power supply for Pedal Board

I’m using some effects pedals when practising, and I also wanted to be able to use them when I’m taking MobFobAmp with me. That’s why I decided that it might be a good idea to have a +9V supply rail with barrel connector (barrel = +) on board. So I bought a very well-known positive linear voltage regulator from the 78xx series called 7809 in a TO-220 design that can supply up to 1.5A of current at 9V, and accepts an input voltage range of 11-35V. The component can be used with minimal additional parts as shown in the image on the right. This way, I can now drive a pedal board from the battery of MobFobAmp!

Piezo Tweeter

When I first used a drum computer with mobfobamp, I felt that the highs just were not enough. While perfect for an electric Bass, the drum machine’s hi-hat, cymbals, and even the snare sounds dull and mumbly. So I ordered a simple piezo driver which I could connect in parallel to the main speaker - this works perfectly well. I added a 2W/320Ohm resistor in series with the driver to get the levels just right to my ears. The result is great TODO: and can be heard here.

MobFobAmp Pros:

Foldable to very compact dimensions
Low Weight: 7.3kg incl. Battery, 6.9kg without
Runs >20h on a battery charge as a practising amplifier and moderate volume
Runs >3h on a battery charge at maximum volume
Sounds really good for Bass, Organ or Acoustic Guitar
Playing low notes down to B0 works fine and sounds punchy
9V DC out for preamps or effects boards
Can be operated in “Bass Reflex Mode” for more Bass

Imperfections:

Without Piezo tweeter, highs are muffled
Low power amp headroom: When turning up the external preamp too much, the power amp will distort notes, especially low ones, first with flutter, then with the typical sound of transistor saturation
Prototype’s rat’s nest wiring is still prone to mechanical damage, so an additional enclosure for the electronics would be nice
Of course, not enough volume/ power to match a drumset or plugged band setup.
Protective grill for speaker would be nice

MobFobAmp - DIY Replica

If you like this idea, feel free to build your own copy! You’ll learn what it takes below. Mobfobamp DIY instructions

Piano by Feliks Weber ↩︎

PC-Upgrade

Schallbert — Mon, 01 Jan 0001

Project Overview

Difficulty: Amateur 3/5
Cost: ~€500
Time required: ~20 hours

Goal and Motivation

In 2006, I treated myself to a gaming PC. I bought an HTPC case. It’s relatively flat and fits in the rack of a stereo system. In 2015, I replaced the components for the first time when the hardware became too slow. By the end of 2025, it was time again: The power supply was struggling with the power consumption of the new graphics card, the hard drives were filling up, and games were no longer running stable.

Upgrades

In short: New hardware is needed. However, this new hardware generates significantly more heat than the previous system. Therefore, some modifications are necessary to optimize airflow. Sounds like a great project with many facets that define this blog: mechanics, electronics, software, and CNC milling.

New Components

To be installed:

650W Enermax power supply replacing 400W
New MSI B550 motherboard
AMD Ryzen 5600X replacing Intel i5-6500
32GB DDR4 RAM instead of 16GB
1TB PCIe SSD instead of SATA SSDs
140mm fan in the 5.25" bay
2x60mm fans in the rear exhaust vents
96mm CPU cooler with a new heatpipe cooler

I’m getting most of the components used. Only power supply, hard drive, and fans are new. However, it quickly becomes apparent that I need to modify the case: The graphics card is so long that it collides with the 3.5" bay, where the hard drives and card reader were previously located. The large 140mm fan only fits in the space where the 5.25" drive bay is located.

Therefore, both bays must be removed or modified with an angle grinder to create space. I also need a mounting bracket for the fan in front of the 5.25" opening. A nice finishing touch for the case would be the ability to automatically control the drive bay door. Then it would only open when the fan is running under heavy load and remain closed during normal operation.

Modifying the Case Lid

In my article on milling SECC steel, I show how to add extra slots to the case lid for better ventilation of the graphics card.

3.5" Bay

{< image src="/assets/images/htpc/floppy-bay-modified.avif" alt=“Image: Cutout in 3.5” bay" position=“right” >} The graphics card is overlaps the 3.5" bay by 1cm. Luckily, the bay can be removed in no time. So I grab the angle grinder and cut a section out of the bracket. Then I drill a few additional holes in the sides of the bay so that my 2.5" SSDs can be mounted on the side of the bay opposite the graphics card. The bay cover also gets another hole for attaching it to the case, since two locking tabs are no longer available due to the cutout for the graphics card.

The card reader uses the full depth of the bay. I’m already worried that I won’t be able to reinstall it at all. I opened its casing as a test, and lo and behold: only the front third is actually occupied by circuit boards. The rear section appears to exist solely for mounting. So I cut off the rear part of the card reader casing and reassemble everything so that it now fits properly in the slot.

5.25" Bay

The bay is tall enough to accommodate two drives. However, I hardly ever use CDs or DVDs anymore — and when I do, an external drive suffices. So, I can remove the empty drive cage and mount a fan in its place. To do this, I’m designing an adapter bracket in CAD that includes mounting points and cutouts for the fan.

Fan Placement

I strategically place case fans to create two separate airflows within the PC case: The CPU power supply airflow is directed to the front of the case via a 140mm fan, while the airflow to the rear of the case is directed to the CPU cooler, from where the warm air is exhausted through the power supply and two additional 60mm case fans.

Airflow to the graphics card is assisted by a 140mm fan mounted outside the case. The graphics card itself exhausts the heated air directly through the case lid.

To connect the external fan, I cut a hole in a slot cover for PCI slots. The socket of a fan connector press-fits into this hole. I secure the socket in the cover with hot glue and connect the fan to an AUX fan header.

Front Panel Control

Now, the front panel should open automatically as soon as the fan spins. It’s more complicated than doing it manually, but this feature really makes the case interesting. So, I remove the panel lock and add a mount for a model servo to the adapter bracket, which will open and close the panel.

Electronics

The servo requires a control signal. I also want to set the flap position, like the fan, based on temperature, so I need to read the voltage applied to the fan. For this, I’m using the microcontroller ATTINY44. I’ve populated the circuit with only the bare essentials:

The microcontroller outputs a pulse-width modulated signal to the servo. Its power supply is buffered by an electrolytic capacitor. I’m routing the power supply for the case fan via two connector strips. There, I tap the signal using a voltage divider and feed it to an analog input. Otherwise, there’s only one additional resistor, which pulls the microcontroller’s RESET input to VCC, keeping it powered on.

Software Requirements

The software requirements are simple and clear:

When the fan is off, the flap should close.
When the fan is running, the flap should open.
To prevent the flap from moving back and forth too frequently, the flap’s state should be changed with a slight delay and only if the fan has maintained its current state in the meantime.

The resulting software is available on my GitHub account.

Software: Main Loop

I keep things simple when controlling the model servo: I use the Arduino Servo library.

The program doesn’t do much. I initialize the servo servoHatch, which is controlled later in the program, define a few variables, and specify which pin the fan’s voltage divider is connected to. The main loop simply reads the voltage at the fan and decides, based on that, whether the flap should move.

I pass the evaluation to the flap control. The program then waits 15ms and starts again.

Servo servoHatch; // create Servo object to control a servo
uint16_t fanVoltage{0}; // variable to read the value from the analog pin
bool open{false};

void setup() {
 pinMode(pinFanVoltage, INPUT);
}

void loop() {
 fanVoltage = analogRead(pinFanVoltage); // reads the value of the fan voltage (value between 0 and 1023)
 open = mapHatchState(fanVoltage);

 
 setHatch(open);
 delay(15); // waits for the servo to get there
}

Software: Hysteresis

Using the mapHatchState function, I define the target state of the hatch based on the voltage at the fan.

If the voltage remains below approximately 4.8V for a sufficient duration, the hatch closes or remains closed. If it exceeds this voltage, the hatch should open. The duration of the state change depends on the voltage itself; at values far below the threshold, the aggregator fills up more quickly, and the hatch state also changes more rapidly.

bool mapHatchState(uint16_t voltage){
 #define maxint 32767 - 1024
 #define minint -32768 + 1024
 static int16_t vAggregate{0};
 static bool hatchOpen{false};

 // Divider VCC 33k T 22k GND, 12V fan will yield 4.8V at divider --> 983 digit
 // So a value of 410 is roughly 5V which allows Fan start + hatch open
 vAggregate = vAggregate + voltage - 410;
 if (vAggregate > maxint) {
 vAggregate = maxint;
 hatchOpen = true;
 }
 else if (vAggregate < minint) {
 vAggregate = minint;
 hatchOpen = false;
 }

 return hatchOpen;
}

Software: Moving the Hatch

The setHatch() function checks the desired state of the hatch and moves it accordingly if it deviates from its current state. A target angle is specified to the servo using write(), which is then set and maintained. The movement is gradual and occurs through iterations of hatchState, as otherwise the hatch would move far too jerkily and quickly.

Your browser does not support the video tag.

Testing the flap control by adjusting the fan's voltage

void setHatch(bool open){
 static uint8_t hatchState{180}; // variable to capture current position, default closed
 static bool wasMoving{false};
 bool isMoving{false};

 if (open && (hatchState > 0)){
 hatchState--;
 isMoving = true;
 } else if (!open && (hatchState < 180)) {
 hatchState++;
 isMoving = true;
 }


 if (!wasMoving && isMoving) {
 servoHatch.attach(portServo); // connect servo on start
 }
 else if (wasMoving && !isMoving) {
 servoHatch.detach(); // disconnect servo on finish
 }

 servoHatch.write(hatchState);
 wasMoving = isMoving;
}

The attach() and detach() functions define the signal output to the servo, internally assigning the microcontroller pin to the servo exactly as it’s wired. In my experiments, I discovered that the servo consumes very little power when stationary in the detached state — much less than when it’s left attached and simply not sending any further movement signals. Therefore, I attach it for each movement and then detach it again afterward.

Result

I now have a nicely organized, modern PC. Using various bloatware solutions (which I uninstall immediately after finishing the work), I manage, after some cursing, to synchronize all the LED colors. A particular highlight is the automatic flap, which activates along with the fan when the CPU load increases.

Privacy

Schallbert — Mon, 01 Jan 0001

Schallbert’s view on privacy

I don’t like being tracked myself, so I’m trying to to author a site for you that saves data to the max. This website is static, so it won’t try loading dynamic contents or third partie’s dynamic plugins except for my channel on the video platform MakerTube. It provides just as many connections to third parties as are absolutely necessary these days to yield a website with a fairly modern and comfortable-to-navigate design. You won’t find the typical cookie banner here because there just are none that I’m collecting.

Privacy summary

This website uses

No cookies
No tracking / trackers
No analytics tools
No login / identification
No embedded content that belongs to third parties with one exception: my channel on MakerTube
All links you find on my page are “pure” links without embedded code or scripts. As long as you don’t click them, third parties will not know.
I do log failed authentication attempts to my Server to protect from brute force attacks. For that reason, I do register IP addresses and time stamps of these failed login attempts and buffer them in rolling logs. They are then used by another piece of software that modifies IPtables on the server’s network layer to effectively ban IPs for a certain time. The gathered rolling logs are deleted automatically every week.
The hoster of this website is Hetzner. I rent a “cloud server” to run my services on their virtualized hardware. Hetzner, as the service provider, collects some data when you visit my site. Please view their privacy statement for more information.

Questions or concerns?

Please contact me via Github Discussions: github.com/Schallbert or using the information on my About page.

Datenschutzerklärung

🇩🇪 only

1. Datenschutz auf einen Blick

Allgemeine Hinweise

Die folgenden Hinweise geben einen einfachen Überblick darüber, was mit Ihren personenbezogenen Daten passiert, wenn Sie diese Website besuchen. Personenbezogene Daten sind alle Daten, mit denen Sie persönlich identifiziert werden können. Ausführliche Informationen zum Thema Datenschutz entnehmen Sie unserer unter diesem Text aufgeführten Datenschutzerklärung.

Datenerfassung auf dieser Website

Wer ist verantwortlich für die Datenerfassung auf dieser Website?

Die Datenverarbeitung auf dieser Website erfolgt durch den Websitebetreiber. Dessen Kontaktdaten können Sie dem Abschnitt „Hinweis zur Verantwortlichen Stelle“ in dieser Datenschutzerklärung entnehmen.

Wie erfassen wir Ihre Daten?

Ihre Daten werden zum einen dadurch erhoben, dass Sie uns diese mitteilen. Hierbei kann es sich z. B. um Daten handeln, die Sie in ein Kontaktformular eingeben. Andere Daten werden automatisch oder nach Ihrer Einwilligung beim Besuch der Website durch unsere IT-Systeme erfasst. Das sind vor allem technische Daten (z. B. Internetbrowser, Betriebssystem oder Uhrzeit des Seitenaufrufs). Die Erfassung dieser Daten erfolgt automatisch, sobald Sie diese Website betreten.

Wofür nutzen wir Ihre Daten?

Ein Teil der Daten wird erhoben, um eine fehlerfreie Bereitstellung der Website zu gewährleisten. Andere Daten können zur Analyse Ihres Nutzerverhaltens verwendet werden.

Welche Rechte haben Sie bezüglich Ihrer Daten?

Sie haben jederzeit das Recht, unentgeltlich Auskunft über Herkunft, Empfänger und Zweck Ihrer gespeicherten personenbezogenen Daten zu erhalten. Sie haben außerdem ein Recht, die Berichtigung oder Löschung dieser Daten zu verlangen. Wenn Sie eine Einwilligung zur Datenverarbeitung erteilt haben, können Sie diese Einwilligung jederzeit für die Zukunft widerrufen. Außerdem haben Sie das Recht, unter bestimmten Umständen die Einschränkung der Verarbeitung Ihrer personenbezogenen Daten zu verlangen. Des Weiteren steht Ihnen ein Beschwerderecht bei der zuständigen Aufsichtsbehörde zu. Hierzu sowie zu weiteren Fragen zum Thema Datenschutz können Sie sich jederzeit an uns wenden.

2. Hosting

Externes Hosting

Diese Website wird bei einem externen Dienstleister gehostet (Hoster). Die personenbezogenen Daten, die auf dieser Website erfasst werden, werden auf den Servern des Hosters gespeichert. Hierbei kann es sich v. a. um IP-Adressen, Kontaktanfragen, Meta- und Kommunikationsdaten, Vertragsdaten, Kontaktdaten, Namen, Websitezugriffe und sonstige Daten, die über eine Website generiert werden, handeln. Der Einsatz des Hosters erfolgt zum Zwecke der Vertragserfüllung gegenüber unseren potenziellen und bestehenden Kunden (Art. 6 Abs. 1 lit. b DSGVO) und im Interesse einer sicheren, schnellen und effizienten Bereitstellung unseres Online-Angebots durch einen professionellen Anbieter (Art. 6 Abs. 1 lit. f DSGVO). Sofern eine entsprechende Einwilligung abgefragt wurde, erfolgt die Verarbeitung ausschließlich auf Grundlage von Art. 6 Abs. 1 lit. a DSGVO und § 25 Abs. 1 TTDSG, soweit die Einwilligung die Speicherung von Cookies oder den Zugriff auf Informationen im Endgerät des Nutzers (z. B. Device-Fingerprinting) im Sinne des TTDSG umfasst. Die Einwilligung ist jederzeit widerrufbar. Unser Hoster wird Ihre Daten nur insoweit verarbeiten, wie dies zur Erfüllung seiner Leistungspflichten erforderlich ist und unsere Weisungen in Bezug auf diese Daten befolgen.

Wir setzen folgenden Hoster ein:

Hetzner Online GmbH
Industriestr. 25
91710 Gunzenhausen
Germany

Tel.: +49 (0)9831 505-0
Fax: +49 (0)9831 505-3

Email: info@hetzner.com

3. Allgemeine Hinweise und Pflichtinformationen

Datenschutz

Die Betreiber dieser Seiten nehmen den Schutz Ihrer persönlichen Daten sehr ernst. Wir behandeln Ihre personenbezogenen Daten vertraulich und entsprechend den gesetzlichen Datenschutzvorschriften sowie dieser Datenschutzerklärung. Wenn Sie diese Website benutzen, werden verschiedene personenbezogene Daten erhoben. Personenbezogene Daten sind Daten, mit denen Sie persönlich identifiziert werden können. Die vorliegende Datenschutzerklärung erläutert, welche Daten wir erheben und wofür wir sie nutzen. Sie erläutert auch, wie und zu welchem Zweck das geschieht. Wir weisen darauf hin, dass die Datenübertragung im Internet (z. B. bei der Kommunikation per E-Mail) Sicherheitslücken aufweisen kann. Ein lückenloser Schutz der Daten vor dem Zugriff durch Dritte ist nicht möglich.

Hinweis zur verantwortlichen Stelle

Die verantwortliche Stelle für die Datenverarbeitung auf dieser Website ist:

Verantwortliche Stelle ist die natürliche oder juristische Person, die allein oder gemeinsam mit anderen über die Zwecke und Mittel der Verarbeitung von personenbezogenen Daten (z. B. Namen, E-Mail-Adressen o. Ä.) entscheidet.

Speicherdauer

Soweit innerhalb dieser Datenschutzerklärung keine speziellere Speicherdauer genannt wurde, verbleiben Ihre personenbezogenen Daten bei uns, bis der Zweck für die Datenverarbeitung entfällt. Wenn Sie ein berechtigtes Löschersuchen geltend machen oder eine Einwilligung zur Datenverarbeitung widerrufen, werden Ihre Daten gelöscht, sofern wir keine anderen rechtlich zulässigen Gründe für die Speicherung Ihrer personenbezogenen Daten haben (z. B. steuer- oder handelsrechtliche Aufbewahrungsfristen); im letztgenannten Fall erfolgt die Löschung nach Fortfall dieser Gründe.

Allgemeine Hinweise zu den Rechtsgrundlagen der Datenverarbeitung auf dieser Website

Sofern Sie in die Datenverarbeitung eingewilligt haben, verarbeiten wir Ihre personenbezogenen Daten auf Grundlage von Art. 6 Abs. 1 lit. a DSGVO bzw. Art. 9 Abs. 2 lit. a DSGVO, sofern besondere Datenkategorien nach Art. 9 Abs. 1 DSGVO verarbeitet werden. Sofern Sie in die Speicherung von Cookies oder in den Zugriff auf Informationen in Ihr Endgerät (z. B. via Device-Fingerprinting) eingewilligt haben, erfolgt die Datenverarbeitung zusätzlich auf Grundlage von § 25 Abs. 1 TTDSG. Die Einwilligung ist jederzeit widerrufbar. Sind Ihre Daten zur Vertragserfüllung oder zur Durchführung vorvertraglicher Maßnahmen erforderlich, verarbeiten wir Ihre Daten auf Grundlage des Art. 6 Abs. 1 lit. b DSGVO. Des Weiteren verarbeiten wir Ihre Daten, sofern diese zur Erfüllung einer rechtlichen Verpflichtung erforderlich sind auf Grundlage von Art. 6 Abs. 1 lit. c DSGVO. Die Datenverarbeitung kann ferner auf Grundlage unseres berechtigten Interesses nach Art. 6 Abs. 1 lit. f DSGVO erfolgen. Über die jeweils im Einzelfall einschlägigen Rechtsgrundlagen wird in den folgenden Absätzen dieser Datenschutzerklärung informiert.

Hinweis zur Datenweitergabe in die USA und sonstige Drittstaaten

Wir verwenden unter anderem Tools von Unternehmen mit Sitz in den USA oder sonstigen datenschutzrechtlich nicht sicheren Drittstaaten. Wenn diese Tools aktiv sind, können Ihre personenbezogene Daten in diese Drittstaaten übertragen und dort verarbeitet werden. Wir weisen darauf hin, dass in diesen Ländern kein mit der EU vergleichbares Datenschutzniveau garantiert werden kann. Beispielsweise sind US-Unternehmen dazu verpflichtet, personenbezogene Daten an Sicherheitsbehörden herauszugeben, ohne dass Sie als Betroffener hiergegen gerichtlich vorgehen könnten. Es kann daher nicht ausgeschlossen werden, dass US-Behörden (z. B. Geheimdienste) Ihre auf US-Servern befindlichen Daten zu Überwachungszwecken verarbeiten, auswerten und dauerhaft speichern. Wir haben auf diese Verarbeitungstätigkeiten keinen Einfluss.

Widerruf Ihrer Einwilligung zur Datenverarbeitung

Viele Datenverarbeitungsvorgänge sind nur mit Ihrer ausdrücklichen Einwilligung möglich. Sie können eine bereits erteilte Einwilligung jederzeit widerrufen. Die Rechtmäßigkeit der bis zum Widerruf erfolgten Datenverarbeitung bleibt vom Widerruf unberührt.

Widerspruchsrecht gegen die Datenerhebung in besonderen Fällen sowie gegen Direktwerbung (Art. 21 DSGVO)

WENN DIE DATENVERARBEITUNG AUF GRUNDLAGE VON ART. 6 ABS. 1 LIT. E ODER F DSGVO ERFOLGT, HABEN SIE JEDERZEIT DAS RECHT, AUS GRÜNDEN, DIE SICH AUS IHRER BESONDEREN SITUATION ERGEBEN, GEGEN DIE VERARBEITUNG IHRER PERSONENBEZOGENEN DATEN WIDERSPRUCH EINZULEGEN; DIES GILT AUCH FÜR EIN AUF DIESE BESTIMMUNGEN GESTÜTZTES PROFILING. DIE JEWEILIGE RECHTSGRUNDLAGE, AUF DENEN EINE VERARBEITUNG BERUHT, ENTNEHMEN SIE DIESER DATENSCHUTZERKLÄRUNG. WENN SIE WIDERSPRUCH EINLEGEN, WERDEN WIR IHRE BETROFFENEN PERSONENBEZOGENEN DATEN NICHT MEHR VERARBEITEN, ES SEI DENN, WIR KÖNNEN ZWINGENDE SCHUTZWÜRDIGE GRÜNDE FÜR DIE VERARBEITUNG NACHWEISEN, DIE IHRE INTERESSEN, RECHTE UND FREIHEITEN ÜBERWIEGEN ODER DIE VERARBEITUNG DIENT DER GELTENDMACHUNG, AUSÜBUNG ODER VERTEIDIGUNG VON RECHTSANSPRÜCHEN (WIDERSPRUCH NACH ART. 21 ABS. 1 DSGVO). WERDEN IHRE PERSONENBEZOGENEN DATEN VERARBEITET, UM DIREKTWERBUNG ZU BETREIBEN, SO HABEN SIE DAS RECHT, JEDERZEIT WIDERSPRUCH GEGEN DIE VERARBEITUNG SIE BETREFFENDER PERSONENBEZOGENER DATEN ZUM ZWECKE DERARTIGER WERBUNG EINZULEGEN; DIES GILT AUCH FÜR DAS PROFILING, SOWEIT ES MIT SOLCHER DIREKTWERBUNG IN VERBINDUNG STEHT. WENN SIE WIDERSPRECHEN, WERDEN IHRE PERSONENBEZOGENEN DATEN ANSCHLIESSEND NICHT MEHR ZUM ZWECKE DER DIREKTWERBUNG VERWENDET (WIDERSPRUCH NACH ART. 21 ABS. 2 DSGVO).

Beschwerderecht bei der zuständigen Aufsichtsbehörde

Sie haben das Recht, Daten, die wir auf Grundlage Ihrer Einwilligung oder in Erfüllung eines Vertrags automatisiert verarbeiten, an sich oder an einen Dritten in einem gängigen, maschinenlesbaren Format aushändigen zu lassen. Sofern Sie die direkte Übertragung der Daten an einen anderen Verantwortlichen verlangen, erfolgt dies nur, soweit es technisch machbar ist.

SSL- bzw. TLS-Verschlüsselung

Diese Seite nutzt aus Sicherheitsgründen und zum Schutz der Übertragung vertraulicher Inhalte, wie zum Beispiel Bestellungen oder Anfragen, die Sie an uns als Seitenbetreiber senden, eine SSL- bzw. TLS-Verschlüsselung. Eine verschlüsselte Verbindung erkennen Sie daran, dass die Adresszeile des Browsers von „http://“ auf „https://“ wechselt und an dem Schloss-Symbol in Ihrer Browserzeile. Wenn die SSL- bzw. TLS-Verschlüsselung aktiviert ist, können die Daten, die Sie an uns übermitteln, nicht von Dritten mitgelesen werden.

Auskunft, Löschung und Berichtigung

Sie haben im Rahmen der geltenden gesetzlichen Bestimmungen jederzeit das Recht auf unentgeltliche Auskunft über Ihre gespeicherten personenbezogenen Daten, deren Herkunft und Empfänger und den Zweck der Datenverarbeitung und ggf. ein Recht auf Berichtigung oder Löschung dieser Daten. Hierzu sowie zu weiteren Fragen zum Thema personenbezogene Daten können Sie sich jederzeit an uns wenden.

Recht auf Einschränkung der Verarbeitung

Sie haben das Recht, die Einschränkung der Verarbeitung Ihrer personenbezogenen Daten zu verlangen. Hierzu können Sie sich jederzeit an uns wenden. Das Recht auf Einschränkung der Verarbeitung besteht in folgenden Fällen:

Wenn Sie die Richtigkeit Ihrer bei uns gespeicherten personenbezogenen Daten bestreiten, benötigen wir in der Regel Zeit, um dies zu überprüfen. Für die Dauer der Prüfung haben Sie das Recht, die Einschränkung der Verarbeitung Ihrer personenbezogenen Daten zu verlangen.
Wenn die Verarbeitung Ihrer personenbezogenen Daten unrechtmäßig geschah/geschieht, können Sie statt der Löschung die Einschränkung der Datenverarbeitung verlangen.
Wenn wir Ihre personenbezogenen Daten nicht mehr benötigen, Sie sie jedoch zur Ausübung, Verteidigung oder Geltendmachung von Rechtsansprüchen benötigen, haben Sie das Recht, statt der Löschung die Einschränkung der Verarbeitung Ihrer personenbezogenen Daten zu verlangen.
Wenn Sie einen Widerspruch nach Art. 21 Abs. 1 DSGVO eingelegt haben, muss eine Abwägung zwischen Ihren und unseren Interessen vorgenommen werden. Solange noch nicht feststeht, wessen Interessen überwiegen, haben Sie das Recht, die Einschränkung der Verarbeitung Ihrer personenbezogenen Daten zu verlangen.

Wenn Sie die Verarbeitung Ihrer personenbezogenen Daten eingeschränkt haben, dürfen diese Daten – von ihrer Speicherung abgesehen – nur mit Ihrer Einwilligung oder zur Geltendmachung, Ausübung oder Verteidigung von Rechtsansprüchen oder zum Schutz der Rechte einer anderen natürlichen oder juristischen Person oder aus Gründen eines wichtigen öffentlichen Interesses der Europäischen Union oder eines Mitgliedstaats verarbeitet werden.

4. Datenerfassung auf dieser Website

Server-Log-Dateien

Der Provider der Seiten erhebt und speichert automatisch Informationen in so genannten Server-Log-Dateien, die Ihr Browser automatisch an uns übermittelt. Dies sind:

Browsertyp und Browserversion
verwendetes Betriebssystem
Referrer URL
Hostname des zugreifenden Rechners
Uhrzeit der Serveranfrage
IP-Adresse Eine Zusammenführung dieser Daten mit anderen Datenquellen wird nicht vorgenommen. Die Erfassung dieser Daten erfolgt auf Grundlage von Art. 6 Abs. 1 lit. f DSGVO. Der Websitebetreiber hat ein berechtigtes Interesse an der technisch fehlerfreien Darstellung und der Optimierung seiner Website – hierzu müssen die Server-Log-Files erfasst werden.

Anfrage per E-Mail oder Telefon

Wenn Sie uns per E-Mail oder Telefon kontaktieren, wird Ihre Anfrage inklusive aller daraus hervorgehenden personenbezogenen Daten (Name, Anfrage) zum Zwecke der Bearbeitung Ihres Anliegens bei uns gespeichert und verarbeitet. Diese Daten geben wir nicht ohne Ihre Einwilligung weiter. Die Verarbeitung dieser Daten erfolgt auf Grundlage von Art. 6 Abs. 1 lit. b DSGVO, sofern Ihre Anfrage mit der Erfüllung eines Vertrags zusammenhängt oder zur Durchführung vorvertraglicher Maßnahmen erforderlich ist. In allen übrigen Fällen beruht die Verarbeitung auf unserem berechtigten Interesse an der effektiven Bearbeitung der an uns gerichteten Anfragen (Art. 6 Abs. 1 lit. f DSGVO) oder auf Ihrer Einwilligung (Art. 6 Abs. 1 lit. a DSGVO) sofern diese abgefragt wurde. Die von Ihnen an uns per Kontaktanfragen übersandten Daten verbleiben bei uns, bis Sie uns zur Löschung auffordern, Ihre Einwilligung zur Speicherung widerrufen oder der Zweck für die Datenspeicherung entfällt (z. B. nach abgeschlossener Bearbeitung Ihres Anliegens). Zwingende gesetzliche Bestimmungen, insbesondere gesetzliche Aufbewahrungsfristen, bleiben unberührt.

Quelle:

e-recht24.de

QR-Codengrave

Schallbert — Mon, 01 Jan 0001

Project stats

Difficulty: hard 4/5
Cost: 0€
Time: ~50h

The idea

A friend asked me to fabricate a coaster for him. A custom logo, some letters. Nothing too interesting. On the backside, he wanted to have a QR-code engraved that keeps a link to his website.

So I had a QR-code created with a standard web-tool and exported as .svg. Unfortunately, in my CAM I’d have to klick every single pixel and create a carve action for that pixel, which would take quite some minutes of my time for click-select-and-enter-value procedures. Plus, I’d have to use a tapered tool and V-carve to get a square pixel which takes much more machining time than a round pixel representation.

That’s where I decided to throw many, many more hours of my time at this problem and come up with a generic solution for QR-code engravings - a piece of software named QR-codengrave.

Getting started

Specification

This is what I wanted the tool to do:

Must-haves:

Take a text input and convert it to a QR-code. No additional config to be done by the user.
Provide an option to manage different CNC tools.
Provide inputs for engrave parameters: engrave depth, in-job rapid movement hover overs, flyover height
Provide inputs for custom workpiece XY0-offsets
Vectorize the QR-code’s pixel output
Have paths created from the vectors, for simplicity start with an inwards spiral
Handover the paths to a module that uses the selected tool’s parameters to create machine paths
Have the machine paths converted to G-code that can later be interpreted by my machine
Thoroughly verify the algorithms with automated unit tests (no crashes in the physical world please)
Provide a graphical user interface to make parameter input as simple as possible

Optionals

Have a Continuous Integration pipeline created both for my IDE and Github Actions
Have a Continuous Deploy pipeline created for my IDE (PyInstaller) and Github Actions (Artifacts, releases)
Add integration tests for the GUI
Optimize the path generation algorithm as “inwards spiral” has a high manufacturing quality and is nice to look at but is not very efficient.

Uncertainities

At the beginning of this project, I had some open questions and risks that could have limited usability of the program or even make the whole project unfeasible. They all turned out to be wrong (thankfully), but still these were the risks that could have rendered all my work useless:

Cameras won’t read round dot-as-pixels QR-codes
The contrast my engraving materials provide is too low so read results could be erratic or not obtainable
Machining time could be very high, making the project uneconomic
Limitations of programming tools or my abilities to code could increase development time to where it’s not worth putting more effort into

That’s where I decided to write unit tests along with the code to get quick feedback about whether my code could do what it should. And I limited the program’s functionality to one engraving strategy and the GUI to only show a preview of the QR-code, not XY-offsets or other fancy stuff for a start.

Programming language

I took some sheets of paper and noted down the essentials.

One module shall take care of creating a QR-code from text input, another one should create vectors and paths to have that QR-code engraved, and a GUI in the middle shall host the text input, a machining preview, and input options for milling tools, engrave parameters, and workpiece offsets.

I had a couple of programming languages that I “speak” well enough to choose from:

C++ with QT framework
Java
Python with Tkinter framework

I decided on the - in my view - most simple solution to have a running prototype as quickly as possible: Python with Tkinter. I know tkinter from an earlier project of mine and although it has its caveats and limitations, it’s still powerful and quick to setup.

Proof of concept

I did some research and found a pretty good open source QR-code library on the web that’s available also in Python. Based on its output, I created a vectorizing algorithm that could interrogate the QR-code library to return values of the QR-code’s pixels in a scan line:

 def _qr_bitstream_from_line(self, line):
 """Creates a bitstream from an input line of a QR-code data representation
 :param line: a QR-code data representation (line of bits within the QR-code)
 :returns bitstream: returns an array of bits reflecting the QR code's state at the respective
 point of the line."""
 bitstream = []
 if line.get_direction() == Direction.RIGHT:
 for x in range(line.get_p_start().x, line.get_p_end().x + 1):
 bitstream.append(self._qr.get_module(x, abs(line.get_p_start().y)))
 if line.get_direction() == Direction.LEFT:
 for x in range(line.get_p_start().x, line.get_p_end().x - 1, -1):
 bitstream.append(self._qr.get_module(x, abs(line.get_p_start().y)))
 if line.get_direction() == Direction.UP:
 for y in range(line.get_p_start().y, line.get_p_end().y + 1):
 bitstream.append(self._qr.get_module(line.get_p_start().x, abs(y)))
 if line.get_direction() == Direction.DOWN:
 for y in range(line.get_p_start().y, line.get_p_end().y - 1, -1):
 bitstream.append(self._qr.get_module(line.get_p_start().x, abs(y)))
 return bitstream

Then, I had to create vectors from these lines, so that consequent pixels with the same values could be merged to a line that’s much quicker to manufacture than multiple dots:

 def _vectorize_bitstream(self, bitstream):
 """Creates a QrLineData object from an input bistream
 :param bitstream: an array of bits = a line of the QR-code
 :returns line_vector: A list of QrLineData objects"""
 line_vector = []
 data = QrLineData(bitstream[0], True)
 for bit in range(1, len(bitstream)):
 if bitstream[bit] == data.get_state():
 data.add_length()
 else:
 line_vector.append(data)
 data = QrLineData(bitstream[bit])
 if not data.get_state():
 data.finalize()
 line_vector.append(data)

 return line_vector

These two methods would now be called by a supervising method that knows a spiral path through the QR-code. My first GUI just had three elements: An entry field for the text to be converted to a QR-code, a button to run the algorithm, and an output canvas where the algorithm’s engrave path would be drawn.

By the way, I used the Turtle module to draw the vectorized QR-code path, a tool that’s been written for college kids who take a dip into program lamguages like Python. It is slow but was easy to setup so that was enough for me to get started with. I hard-coded it in to save time, and my loops are more C-ish than pythonic but nevermind.

 for vect in self._spiral_path[i].get_z_vector():
 if self._stop_draw:
 break
 length = vect.get_length()
 if vect.get_state():
 self.turtle.down()
 self.turtle.forward(self.pen_size * length)
 self.turtle.up()
 else:
 self.turtle.forward(self.pen_size * length)
 self.turtle.right(90)

This very simple concept looked promising to me but I had to tweak it a couple of hours because my initial output looked like this:

Your browser does not support the video tag.

QR-codengrave GUI test

Still, after a while I was able to reliably draw a (21x21 Pixel) QR-code on screen, and use my smartphone’s camera to yield the webpage link. 🥳 Yeah!

Your browser does not support the video tag.

QR-codengrave vectorizer working

Unit tests

The “platform” parts - vectorize_qr and machinify_vector modules both contain a lot of verifiable calculations and business logic. That’s why my tests with Python’s standard unittest module are quick and easy to write. This is an example where I verify that the line class I created correctly calculates a length (n+1 style):

class TestLine(unittest.TestCase):

 def test_line_x_calculates_valid_length(self):
 line = Line(Point(0, 0), Point(5, 0))
 self.assertEqual(6, line.get_abs_length())

But it quickly became more complex when I wanted to test whether the output of the QR-Code generator library is handled correctly in my program. For that I had to inject the library into my class under test so I could sneak a mock implementation in that’s then interrogated from within the test.

This pattern of dependency injection is very similar to how it’s done in C++ where I’m much more proficient (see a project here or another one there, for example), only that in Python far less boilerplate code is needed to get there. Some internet research helped me write the following test case:

 def test_bitstream_qr_input_loop_xpos_correct(self):
 mock_qr = QrCode.encode_text("schallbert.de", QrCode.Ecc.MEDIUM)
 mock_qr.get_module = MagicMock()
 cam = VectorizeQr(mock_qr, 1)
 cam._qr_bitstream_from_line(self.test_line_x)
 mock_qr.get_module.assert_called_with(20, 0)

I inject the “real” QrCode library but with one mock method which is of interest to me, get_module(). In the test, I’m asserting that it is called with the expected parameters from within my class under test.

With these tests in place, I can be pretty sure that the building blocks of my algorithm work as intended, and that regressions are quick to be found.

The long way to Minimum Viable Product

Boosted by this quick win, I went all-in. I had to design a tool selector before implementing the G-code generator because I’d have to have the tool properties available before setting dimensions and tool speed / feed for the QR-code engraving action.

After some unsuccessful tries in the main window, I decided the tool config window should be a dialog instead (that Tkinter calls Toplevel) so tool parameters can be entered and sent back to the main GUI once complete. I made the dialog modal with

 self._dialog.grab_set()

So it would always have focus to avoid trigger several instances of this window. I keep the configured tools in a numbered list so once fed, the application would provide the list so the user could select the required tool for the job from a dropdown menu.

With this step, I had to implement two things:

Checks for value type and range
A Persistence module so that entered tools would survice an application restart.

Value type and range checks

This could be accomplished using an event hook within the Entry Box widget like so:

 reg = config_tool_frame.register(validate_number)
 tool_nr_entry.config(validate="key", validatecommand=(reg, '%P'))

And then handing over the entered value to a simple validator that just tries to cast the input value into a float and returns False if that generates an error,

def validate_number(entry):
 """Helper function that is used in validators to check
 that the entered keystroke is a number"""
 if entry == '':
 return True
 try:
 float(entry)
 return True
 except ValueError:
 return False

Range checks would be made not when something is typed into the fields, but only when the OK button is pressed to relay the entered data back into the application. Once out of range, a warning Popup (tkinter.messagebox module) would appear pointing to the possible issue.

Persistence

The Persistence module was a bit less easy to implement. I decided on creating an item what C++ guys would call a static class, i.e. a class that doesn’t have to be instantiated to call one of its methods. This class just has two methods called load and save that do what their name suggests using Python’s pickle for serialization of objects that I wanted to be persisted. In Python, I used the @classmethod decorator to indicate the static nature of that class.

Have a look at how simple data serialization can become with the right tools:

 with open(app_persistence_path, 'wb') as file:
 pickle.dump([cls._tool_list,
 cls._z_params,
 cls._xy0],
 file, protocol=2)

When this reliably worked, I added an engrave parameter configuration widget.

Engrave parameters

To generate G-code from my QR-code data, I not only needed tool and speed / feed data, but also I’d have to know how deep to engrave, how far above the workpiece’s surface the tool should hover for rapid moves, and what safety height to use for the CNC to return to origin or home.

It is created in a similar manner to the tool_configuration dialog and uses the same mechanisms for type and range checking. The only difference is that it’s launched not with a button click, but via click on a label that shows the current engrave parameter value in the main window.

Workpiece XY Zero offset

The workpiece offset config dialog features some radiobuttons to select most common workpiece origins, and provides a custom option where the user can enter values for workpiece’s X and Y axis zero. A function in the background calculates the required offset relative to the engrave spiral’s starting point.

Normally, I wouldn’t want the GUI to do calculations or a lot of branching logic because it’s harder to verify as test fixtures become larger and more difficult to maintain. Still, in this case I decided to do the offset calculation with help of a lookup table within the GUI module to avoid having to communicate through all GUI modules to platform and back (4 classes total comms length).

 def _get_xy_offset(self, offset):
 """calculates an XY coordinate offset from a preset point, taking the selected tool diameter into account.
 :param offset the selected offset from the above enum class
 :returns a XY Point where XY0 is assumed for the engraving."""
 qr = self._qr_dimension
 d = self._tool_diameter
 offsets = {Offset.CENTER: Point((d - qr) / 2, (qr - d) / 2),
 Offset.TOPLEFT: Point(d / 2, -d / 2),
 Offset.TOPRIGHT: Point(d / 2 - qr, -d / 2),
 Offset.BOTTOMLEFT: Point(d / 2, qr - d / 2),
 Offset.BOTTOMRIGHT: Point(d / 2 - qr, qr - d / 2)
 }

 if offset in offsets:
 return offsets[offset]

Initial release V1.0

The initial release is a minimum viable product. It does its job but still has some flaws (like throwing an error when there’s no tool selected in the drop-down) and imperfections (like drawing a QR-code is really slow, re-drawing tilts the QR-code a bit etc.) but it should just work, helping the user avoid the worst blunders on their way.

I have tested the execution on my CNC machine and at least my phone is able to navigate to a website just by holding the camera at the workpiece 📱

Automated integration tests for the GUI: frustrating work

When my initial release was out, I wanted to add some tests related to the GUI. As I didn’t manage to have absolutely no business logic there, I at least wanted to verify the range checks as well as the communication with the platform to ensure that it all works as intended.

Stuck in Messagebox

For that, I wrote some tests that ran on my local machine with the disadvantage that when the test hit a warning or error dialog messagebox.showinfo / messagebox.showerror, a manual user interaction was absolutely required to have the window closed and the test to continue.

I didn’t find an easy way to circumvent that issue as the messagebox acts as an independent widget that cannot easily be tampered with. Even my tries to invoke a keystroke as a side effect from within the test did not prove to be fruitful (and, in the end, wouldn’t have been a clean sollution, I suppose).

My solution after a couple of hours cursing under my breath is that the messagebox widgets are injected into the GUI classes that use it:

@patch('src.gui.gui_tool_manage.GuiToolManager')
 def setUp(self, mock_guitoolmanager):
 tk.Tk() # required to have tk variables properly instantiated
 self.mock_guitoolmanager = mock_guitoolmanager
 self.mock_msg = MsgBox()
 self.mock_msg.showinfo = MagicMock()
 Persistence.set_mock_msgbox(self.mock_msg)
 self.config_tool = GuiConfigureTool(self.mock_guitoolmanager, self.mock_msg, {'padx': 5, 'pady': 5})

This way, I can throw mock_msg into the gui class so not the real showinfo was taken but instead the mock would know whether it has been called as intended. I could add it to the test fixture so my test code itself could stay DRY.

It adds more arguments to the constructor and delegates the instantiation to a higher level class, but still the usage inside the class under test stays trivial.

Static class issues

As the persistence class I mentioned above is static in a way that it doesn’t require instantiation, I had the next issue built in my code right from the start: An integration test would now try to write to an actual persistence.dat file, a dependency that I didn’t want to have, especially not in the continuous integration environment.

So I moved Persistence up the stream into the main GUI file that wouldn’t be tested following Steven Anderson’s splendid blog post. The main GUI should only contain declarative code anyways, and the loaded data is then cascaded into the classes that use it instead of the persistence class that actually does the load/save operation.

Future releases

This is what I’m planning for future releases:

Drawing speed of the QR-code must be increased
Graphic bugs (scaling, tilting) and blurry edges of drawing screen to be fixed
CNC paths to be optimized for lower manufacturing time
Bugfixes

At least if there are any; let’s see how much I’ll be using this tool. The things that annoy me most will likely be fixed first.

Ah, one last thing: be rewarded for following through on this article. You can download your copy of QR-codengrave for free to play around with, and for home use. If you’re planning for commercial use, please let me know upfront.

Seifenbutler

Schallbert — Mon, 01 Jan 0001

Project stats

Difficulty: Medium 3/5
Cost: ~250€
Time: 1+ year

The story of Seifenbutler

Accompany me here over a period of more than a year, in which I am constantly working on one project. It goes from an idea with a few requirements to pre-production readiness. I explain my prototype work, go into a little detail about the many setbacks and finally come closer to what I think is a good product.

Idea

May 2022

I recently switched from using liquid shampoo in the shower to solid shampoo and soap. As I didn’t have a tray for the soap, I showered with it. It used up more quickly, didn’t dry for days and stuck to the edge of the shower.

I didn’t want to buy a normal soap holder that would require drilling holes in the tiled walls. So I decided to design a clip-on soap holder for the shower rail. I would mount the piece above the shower head so it wouldn’t be exposed to more water than necessary.

The soap holder should also be easy to fit without tools.

V1-V3: First drafts in HPL

July 2022

So I designed my own holder. I wanted a simple, round design with a clamp hole to attach to the shower rail and a sunken area to hold the soap or solid shampoo. A few slots in the base would allow water to drip off and the soap to air dry when not in use. I wanted to add a fastener inside the piece and a nut on the inside to reduce the diameter of the hole for the shower rod and clamp seifenbutler - as the piece is called - in place.

To make this possible, it made sense to produce two components - an upper shell that holds the soap in place and provides space for the mounting mechanism and a lower one that has drainage slots for excess water and holds the other half of the mounting mechanism.

FreeCAD: Sketches

FreeCAD artefacts are organized in a certain way to simplify the handling of different views or to prepare for later two-sided editing using the clone function. Therefore, I first create a series of parts in ‘Parts Design’ mode. Sketches can be grouped into body elements, which in turn are arranged in part elements.

So I create a part (yellow element) that contains “sketches” - all the original drawings (red elements) of all the bodies (blue elements) of the part(s) to be produced. These drawings should be fully constrained. They cannot contain any information on positioning relative to other components. This means that the sketches are all in the same place. Therefore, all inactive sketches should be hidden to avoid confusion.

(A more in-depth description of my FreeCAD workflow can be found in this blog post.)

FreeCAD: Parts

Then I create parts that later contain the components to be cut, e.g. “clampingHoles”, “soapButler”, “wallHolder”. They each contain a subset of sketches, but no information on the final positioning. Finally, I create a part for each job that my production has to perform, e.g. job00_clamping, job01_upperMill, job02_3dCut, job03_lowerFinish. These clone body components but add placement information, include mirrored parts for 2-sided milling or provide an outline to set the XY-0 correctly.

The advantage of such a project configuration is that if an original sketch has to be changed for any reason, all cloned parts are automatically changed as well. With clever organization, it is also possible to simply change the dimensions of the parts in the table that then adjust the entire part geometry accordingly without help of the user.

Machining

For my first prototype, I chose high pressure laminate as the material as it is very durable, can get wet from time to time and looks pretty good. For machining, I used low helix endmills to keep the vibration under control when cutting the tough HPL material. I opted for coated single flute carbide bits that I bought some time ago (for cutting aluminum). I used eccentric clamps to hold the part in place and only required two additional hold-downs to keep the part on the machine bed.

I had an error in the CAM program that took me a few minutes to fix: I had set the toolpaths for the spiral inward instead of outward. The problem with this is that the spiral becomes increasingly spongy and unstable as the cut progresses. I was only able to keep the vibrations in check by frequently interrupting the milling process and applying tape to the areas where the cut had already been made.

Rating

Production of my first prototype went quite OK. The machine was humming confidently, and apart from a few minor errors (fixing holes in the lower part too narrow, length of clamp fixing incorrectly specified) I was reasonably happy with the result.

It took me about 20 minutes to release the part and remove the taps. This seemed to be clearly in need of improvement.

V4: Failure with cheap HPL

October 2022

I’ve never used the chamfer function in my CAM before and thought this would be a good opportunity for a test as the part corners were quite sharp on my first prototypes. As I only use a 4mm end mill, the 90° 10mm chamfer cutter seemed too big, especially as I had to offset outwards to avoid wearing out the cutter’s tip. I therefore chose a 30° tapered bit with only 6mm diameter, which quickly turned out to be a mistake as it hardly softened the corners.

I also forgot that only the outside of the parts would benefit from a beveled edge. I, on the other hand, applied the chamfer to the inside, which was detrimental to the finish quality. I also found that the new HPL sheet I used to make the prototypes was not completely flat, so some parts of the outline were not fully milled through. This in turn led to a lower quality and I had to do considerable manual rework.

V5: Corrections in CAM and better processing quality

November 2022

The “free arm clamp” design with open fastening element is not so nice to look at, I guess, because it is asymmetrical. It would also be difficult to clean because of the narrow gap. After some thought, I came up with the solution of a “center clamp”: the fastener would press against a piece of rubber that rests directly against the shower rail to hold the soap dish in place.

Chamfers

After updating the drawings accordingly, I created another mirrored part in FreeCAD in “Draft” mode. This part clones the original part at the outline and is intended for two-sided milling, rotating the part around its top left edge (XY zero). In this way, the chamfer can be implemented correctly if I can ensure that the XY zero point does not move.

This worked very well in practice, and I was also able to automate the drilling of countersunk holes for the lower fasteners, which increases the build quality. The result can be seen below.

Rating

Unfortunately, after this fifth piece was finished, I noticed that the melamine coating was slightly chipped at the cut edge. I also noticed the smell of burnt material during processing. A quick look at the cutter revealed the cause: it had become blunt after just one hour of use, despite the coating. This makes working with HPL quite expensive and frustrating. In addition, the cycle time for this version was quite long: 15 minutes for one part. That was never going to be economical¹.

That’s why I tried to find alternative materials for Seifenbutler. In the meantime, the prototypes should be tried out in the shower for a few months to gather long-term experience. Is HPL suitable for use in damp areas? Does the water creep through between the two layers? Is it easy to clean?

V6: Failure with birch multiplex

May 2023

In my search for new materials, I procured waterproof wooden boards, glued together in multiplex method. As it was considerably thicker than the previously used material (15mm), I simplified the design into a single-piece version with a grub screw at the side to clamp it to the shower rail.

I would drill the hole for the grub screw by hand after production on the CNC.

Unfortunately, however, the material on CNC turned out to be unsuitable for milling the spiral. I got numerous tear-outs and had difficulty holding the workpiece down. The contours of my Seifenbutler were simply too fine for the cross-glued material. Maybe the wood’s fiber length was too high, too.

The five machined parts went straight into residual waste.

V7: Failure with Aluminium-Composite material

Juli 2023

With this design, I returned to my two-component setup and wondered whether Aluminium composite would be a suitable material for Seifenbutler. It is known for its durability and is often used outdoors, at least.

Without further ado, I drew up some new sketches² that tried to take properties of the new material into account. The light material would finally make it possible to use magnets to hold it in place, so I made appropriate countersinks in the frame.

I had the idea of creating a part from each side of the shower rail and then holding it together using the magnets - this finally got rid of the annoying clamping screw and gave me a very simple design.

Unfortunately, however, the concept turned out to be completely unsuitable. The holding force of the magnets is not high enough and the clamping only works by tilting the soap holder forwards, which gives a strangely crooked impression on the shower rail.

So: keep tinkering!

V8: Switch to acrylic glass, chuck

July 2023

After many hours at the computer for the design, I came up with a new clamping principle, copied from the drill chuck of my hand drill: three movable clamping devices mounted at a 120° angle - brought together by means of a cable tie - would hold shower rails of any diameter known to me.

Manufacturing

I encountered various difficulties in production, mainly caused by per design small components, which also had to have notches to guide the cable ties. In addition, I realized that designing and producing smooth fits without noticeable play was a major challenge for me and that it would take a lot of unsuccessful attempts before I reached my goal.

However, by using different sized (2mm, 4mm) milling cutters and optimizing the milling sequence, I was finally able to produce the parts successfully and in good quality. See my article on milling small parts for reference here. Only the spiral is still giving me a headache, because even in acrylic glass it would prefer to avoid the milling cutter. As a result, its edges are quite sharp and uneven.

Fortunately, it was easy to assemble - It only needs one screw and some plastic glue.

Test

I had this variant in my shower for a few months as a test. As the screw was only galvanized, it unfortunately started to corrode. Note: use stainless steel.

The stability of Seifenbutler could still be improved. The cable tie leaves a little play, so that with little effort, inclinations of around 10° to the vertical of the shower rail are possible, causing a lot of creaking between the upper and lower parts. This needs to be improved.

I would also like to simplify the clamping mechanism again.

V9: Cast acrylic glass, removable soap tray

July 2023

This evolution step holds upper and lower shells together at the front with magnets that can be invisibly sunk into the material. There are two inserts in the clamping area at the back, which fit into a suitable recess in Seifenbutler like a drawer and hold it together.

It took me several prototypes to get the tie guide to the point where the cable tie is simply inserted into the back of Seifenbutler, wraps itself around the shower rail and finally reappears where it is ready to be tied down.

This is the first design that meets my requirements in terms of holding power and ease of assembly. Unfortunately, the production is complex and the tolerances have to be kept low, as the soap holder either slips out of the clamp too easily or can hardly be fitted because the fit is too tight.

Your browser does not support the video tag.

Seifenbutler zip tie demonstration

V10: Improved manufacturing process

August 2023

With a few optimizations, this version corresponds to V9. I achieved better repeatability in manufacturing and I could be a little more generous with tolerances for fits.

With this design, I obtain first feedback from a couple of friends. The unanimous opinion: cable ties look like a makeshift solution so customer acceptance might be low.

So I head back to the drawing board.

V11: ‘L-nose’ plug-in system in HPL

August 2023

Another attempt in HPL, this time with multi-tooth cutters. They rasp the material rather than cutting it and are designed for processing abrasive, fiber-rich materials such as CFRP/GRP. However, the high toughness of HPL combined with poor heat dissipation means that the milling cutters become completely blunt after just two prototypes have been produced. So they are not a solution for efficient work in HPL for me either.

The design is again multi-part and achieves the clamping in its last update via a rear-mounted screw, which is tightened after the front and rear parts have been put together. This makes the soap holder longer overall, which I don’t find ideal in terms of design.

On the other hand, I think I now have a good handle on the manufacturing quality. The cuts are clean and the quality of the edges is also very pleasing.

Due to the rapid cutter wear, it is very difficult for me to produce a slight oversize fit for the plug-in connection. It happens all too easily in my attempts that the cutter loses diameter too quickly and that parts cannot be inserted into each other at all or only with brute force.

Perhaps I can solve this problem by using a different fitting geometry?

V12: ‘Brio’ plug-in system

September 2023

Here I use acrylic glass again, this time in the extruded version. This way, thickness tolerances are very small compared to the cast version and my Z-zero points always fit. This is great for applying chamfers, which are finally the same width on every part and give the material a high-quality finish.

Now I also have a solution for the spiral: double-sided processing. This increases the workload for me, but I’d rather spend more time at the machine than be annoyed about poor quality later on.

My procedure: I mill the spiral almost all the way through and leave only a delicate 0.3mm. I then use the chamfer cutter on the back and cut the spiral from inside to outside. I also secure the underside with adhesive tape so that my vacuum table can hold the spiral securely and vibrations are avoided.

The design includes a knurled screw so clamping can be done without tools. As the clamping exerts strong tensile forces on the connectors and they therefore have to be positioned in the axis of the shower rod for stability reasons, I also have a slight tilting problem here. This problem increases the smaller the diameter of the shower rail is.

I was able to avoid tilting with another variant, but then the spigots are no longer covered. This in turn means that shower water can now get in there and leave residues.

I’m almost there. Nevertheless, there may still be something I can improve.

V13: Automatic clamping and zero series production

October 2023

In version 13, I try to bundle the advantages of the previous versions V9-V12 while avoiding the disadvantages (tilting, cable ties, assembly with tools, unstable) as much as possible.

New clamping concept

I am therefore pursuing a new clamping concept. I still want to take most commercially available shower rods into account during installation and therefore need to clamp securely between 18mm and 25mm. From the previous versions, I have also realized that additional pressure must be exerted centrally behind the shower rail to prevent Seifenbutler from tilting.

I am also trying to realize the locking mechanism around the shower rail by means of a rotary movement. Pivot point is the center of the spiral, which requires three additional screw connections around the circumference of Seifenbutler. With V13.9 I finally achieve the breakthrough.

The automatic clamping mechanism is activated by pushing upper and lower shells together so that Seifenbutler adjusts continuously to the diameter of the shower rail. It is locked in place by a knurled screw on the underside, where an inclined plane on the catch hook prevents unintentional opening.

Manufacturing

Unfortunately, this design is very complex. It consists of five components made of acrylic glass, five screws and a knurled screw with spring washer. It takes quite some time to assemble.

The work on the CNC takes over 10 minutes per part, which is a lot. I also have to use consumables due to the required workpiece hold down.

Nevertheless, the manufacturing quality is higher than ever before and Seifenbutler is now child’s play to fit without tools and within a short time.

Conclusion

January 2024

Looking through my CAD files, I have planned a total of 42 versions of my soap holder so far, divided into 13 designs, some of which are very different. 21 of these versions have actually made it onto the CNC in the end and 5 of them have been produced in larger quantities than 3.

I have four Seifenbutlers in my shower for endurance testing and they are now used by the whole family.

I can’t really count the number of hours I’ve spent on the subject so far - but it’s been many. I’ve learned a lot about materials and their suitability for different part geometries, absorption of forces, their machinability and the necessary cutting values on my machine. And also that it’s worth pursuing things with determination over a long period of time. Not in monetary terms, but definitely for me as a person.

I don’t want to rule out making further optimizations to the soap butcher - but first I want to see whether it is marketable as a niche product in the near future.

I give a deeper dive into the challenges of machining HPL in this blog post. ↩︎
As you can see from the CAD images from here on, I changed my drawing software (from FreeCAD to CADasCAM). Despite some disadvantages, it is easier to use for 2D drawings and I get results faster. One big downer: CADasCAM is not open source 😥. ↩︎

Spindle upgrade

Schallbert — Mon, 01 Jan 0001

Project stats

Difficulty: Expert 5/5
Cost: ~2700€
Time: ~20h

Upgrading my CNC's spindle↗

In contrast to my statements here I decided to invest in a High-Frequency spindle. I went through a lot of reading, building and testing effort before I could finally create chips with the new setup.

Why I upgraded

There are strong and weak arguments for such an upgrade. Long story short: My arguments were rather weak so I could have sticked to the router motor I already had collected experience with. The decision to upgrade anyways was on the non-rationale side.

Strong arguments to upgrade 🥰

More power: This is not supposed to happen again, and I can run high feed rates with confidence. As a result, machining time reduces for projects that require removing loads of material.
More low-end torque: With a face mill cutter, the router motor tended to overheat as the bolt-on fan wouldn’t provide enough cooling at low rotation speeds.
Vibrations: I hope that the vibration issue is gone now because both spindle, its attachment to the Z-axis, and vacuum table provide the CNC with higher stiffness.
Likely higher precision of cut on the spindle due to higher-grade bearings and more rigid design

Weak arguments to upgrade 🤔

I was very interested in variable frequency drive and spindle technology
I liked the looks of the Spinogy spindle.
I was in contact with Spinogy for half a year discussing possible configurations and the adaptation to my machine and I didn’t want to let them down
The VFD has internal safety systems that reduce probability of damaged workpieces
The HF-spindle is more quiet than a router motor

Arguments not to upgrade 🤨

Low value add for money: Versatility of the machine is increased by just a little while upgrade costs are really high
Spindle warm-up takes 30 minutes, a considerable amount of time when you’re not using the machine 24/7
Steep learning curve required for configuring VFD/Spindle, >400 pages of VFD and spindle manuals to read, circuit layouts to plan, and a switching cabinet to build.
The CNC frame I have is laid out for light to medium usage. Turning up the feedrates because of the new spindle’s capabilities could overburden frame and structure, especially the machine’s weak spot, the Z-axis.
The HF-spindle weighs 3x more compared to the router motor.
A VFD system makes the whole CNC much more complex and adds a multitude of paths for disturbances.

All in all the spindle is overpowered for the rest of the machine and the benefit for adding the spindle likely will be low. You see this was an emotional decision. I hope I won’t regret it.

Auto tool change?

If I had a well-running woodworking business with clear scope and just a few endmills in use, I’d have decided to invest the extra money.

For my prototyping usage on the other hand the hypothesis I made in the past still holds true. I’m using a lot of different endmills as I’m working with many materials that each require a unique set of tools. The cost of steep taper toolholders I’d require to cover my work would have exceeded the cost of the spindle.

Instead, I bought some more spindle nuts. This way, I can keep the most-often used collets attached to their nuts, reducing the time required for a tool change.

Cost 💶

I made some other decisions to both match my way of intermittently operating the machine as a sideline and to keep cost down. In my spindle configuration, I selected the 1.5kW low power option and did not raise specs of the bearings so the machine is limited to 30kRPM. I chose forced air cooling over the water-cooled option and did not invest in a continuous temperature monitoring system.

Still, total system cost are more than 2x spindle cost. Note that although the VFD is not too expensive, shielded motor wiring, connectors, torque wrench, line filter, collets/nuts, the switchbox and many additional, small parts add up to that high of a number.

Information phase

To get the new spindle running I had to plan, build, and configure a complete subsystem that involves spindle drive, cooling, auxiliary power supply. Driven by the project complexity, I executed my work in different phases that would build on top of each other. Starting with the information phase, let’s go through some of the questions I had to find answers to.

Variable Frequency Drive

As I wrote in this article, a VFD is used to control rotational speed of a HF-spindle. The motor is called like that because it operates well above grid frequency (50Hz in my region) to reach higher rotation speeds as required by the application, especially when working with smaller endmills.

When I took a deep dive into that topic for my new spindle application, I had a phone call with Mr. Wagner from Omron (VFD manufacturer) to answer some of my questions.

`U/f characteristic` or `sensorless vector control`?

Selecting U/f characteristic will make the VFD reduce voltage along with the frequency as long as the motor operates below its corner frequency (which for my machine is 400Hz). This is because the inductance of the stator wires reduces with lower frequency. That in turn increases the current flow through the stator, and along with it torque and power dissipation rises. So in the end, by applying the U/f characteristic, current and with it the torque is kept constant. At very slow speeds though, the inductance becomes so low in comparison to the stator resistance that torque decreases over-proportionally which makes this option a bad choice when operating in that low-speed area. At or above the motor’s corner frequency on the other hand, voltage is kept stable and just the frequency is increased which reduces torque.
With sensorless vector control, the VFD uses a mathematical model of the motor it drives to calculate optimal switching times to always achieve nominal (or even above) motor torque independently of the motor’s rotational speed. It is especially useful for applications that require heavy starting like elevators etc. and often requires the user to perform a VFD auto-calibration run with the motor connected.

Here’s a link to somebody’s youtube video in which the two options are compared at low speeds. Here’s another one that demonstrates how the VFD even goes beyond nominal motor torque when load is applied.

As most of the endmills I use demand a speed of 15000rpm or higher, I likely won’t feel the drawbacks of the U/f characteristic. Plus, on my VFD, the sensorless vector control option is only available up to 400Hz so I would only be able to use 80% of the spindle’s speed range. Easy decision: U/f it is.

freewheeling, brake resistor usage or energy recovery system?

For braking the motor, the VFD can be configured with different options. When allowing the motor to freewheel in standard setting, it may take a long time until which the spindle has stopped from full speed.

When a brake resistor is configured like I do with my spindle application, braking down takes two seconds only. I also configured the DC brake to completely bring the shaft to a halt.

For more powerful motors or applications that involve movement of high masses, an energy recovery system might make sense. It is connected to the VFD’s intermediate circuit and is able to supply power generated through the motor’s excess motion energy back to grid. To use this feature, an additional electrical device is necessary and compabibility of the VFD’s intermediate circuit with that device has to be made sure. This adds high cost to the system and only pays off when large amounts of energy can be recuperated.

Which modulation freqency / clock frequency should I use?

The clock / modulation / carrier frequency is the rate at which the power transistors within the VFD switch to provide an output voltage for the motor that resembles a sine wave. On my VFD, it can be configured in a range of 2-15kHz.

To select an optimal frequency, a balance is to be found between multiple factors:

Transistor switching losses rise at higher frequencies.
Transistor on-state losses rise at lower frequencies.
The human ear is especially sensitive in the range of 2-5kHz.
Resonance frequency of the motor system should be avoided.
Electromagnetic interference emissions rise at higher frequencies.

Of course, if your motor manufacturer makes a modulation frequency recommendation, you should follow that one. I played around with different values and chose 5kHz as a good compromise between the above factors. I also felt it sounded better than 6 or 4 kHz on my machine.

Can I control the coolant fan based on spindle temperature?

Yes, that is possible. All you need are a continuous (non-switching) type of temperature sensor, e.g. PT100 or other thermocouple, a measuring transmitter, and a free analog input of your VFD that can be configured to on-off control one of the free digital output ports that then switches a relay the cooling fan has to be connected to.

My motor, unfortunately, is equipped with a switching-type thermistor that connects to the VFD via a temperature alarm input. Thus it can shut the system down once an over-temperature event occurs but is not suitable for a continuous temperature monitoring/control of the cooling fan.

Is `18V` enough to detect `HIGH` on a `24V` digital input?

Yes, on my VFD I was able to feed a lower voltage than the targeted 24V to a logic input and still make it detect HIGH level.

Can I connect VFD logic ground to my DC supply ground without issues?

Yes. I did and there were no issues. I even had to do so because my ‘Run’ input is potential-free so I had to connect it to the DC power supply’s positive voltage rail over a relay contact. Also my numerical controller’s analogue output voltage to control spindle speed is forced to use the same ground reference as the VFD, so it’s also connected to the VFD’s logic ground.

What do I need to consider regarding electromagnetic compatibility?

Use a line filter between mains switch and VFD.
Use shielded cables for the motor you connect.
Keep motor cable as short as possible.
If you have to use long cables, add a HF noise filter.
Have the shield connected to the VFD’s protective earth panel along with the motor’s protective earth leads.
Make sure the shield establishes conductive contact to the cable connectors.
Perform a continuity test.
Make sure the switchbox has one central protective earth clamp where every device within the switchbox connects to.
Make sure your supply’s protective earth is connected to that clamp, too.
When wiring the switchbox, position motor cables as far away as possible from low-voltage signal wires.
Use a metal switchbox housing (connected to the central earth point, of course) or at least a metallic carrier plate you mount all devices to. This will reduce radiated electromagnetic interferences.

Spindle

To select a matching spindle for your machine and application, there are a few points to consider - independently of the motor type.

Power

The spindle power maybe is less important than you might think. I’ll give some examples you can follow to choose a good match.

<=400W HF-Spindle or <=800W router motor: You have a hobbyist machine and do not require to run it blazingly fast. It is fine for you to go shallow or medium depth per pass on hard to cut materials like aluminium or HPL. Your machine frame weighs less than 60kg and is relatively big for its weight, e.g. workbed surface of >=0.5m².
<=700W HF-Spindle or <=1200W router motor: Your machine weighs more or is more stiff than the above option. You use bits bigger than 6mm in diameter a lot, or have to machine parts with face-milling cutters often. Your Z-axis is strong and you feel your machine is bored when running through hard wood at 3000mm/min, Z+6mm with a 6mm cutter.
1.1kW / 1.5kW HF-Spindle: You have a compact machine (around 500x500mm travel, weight: 100kg) and do want it to finish its jobs quickly, cutting aluminium or wood at higher feed rates. When working with wood, you use roughers a lot to save time. Your machine can handle higher accelerations and rapid movements (e.g. 1000mm/s² and 300mm/s).
>=2.2kW: You left the hobby sector - likely forever - both with your machine and the order bank that has piled up on your desk. You have a workshop, a professional dust collection, and an heavier machine you trust to run all day long without intense supervision. You might consider this blog as “too basic” and head off to new shores where people can really help you as a professional.

Weight

Weight scales with power. An 800W router motor can come at below 2kg including mount while a 2.2kW HF-Spindle with automatic tool change might be at or above 6.5kg without tool holder. Footprint and cables also tend to become larger. A high weight can negatively affect light machines, especially. In an extreme case, the Z-axis might start moving down when the stepper motor is not energized and machine acceleration, especially at the portal movement axis, might have to be reduced so no steps are lost. Also keep in mind that heavier machines tend to be bigger, so the cutting forces can have unfavorable effects on the Z-axis due to increased leverage.

2-pole or 4-pole design?

If you need a lot of torque, especially at low speeds, chose a 4-pole design. Keep in mind that you will require double the frequency to have it turn with the same RPM as a 2-pole motor. If you often run your machine at high RPM instead, consider buying a 2-pole spindle.

Water or air-cooled?

This again is more a question of cost and professionality than anything else. Of couse, bolt-on fan self-cooling works fine. Forced-air cooling comes a little more expensive but is likely more quiet, its main advantage being that the spindle does not get too warm independently of its speed of rotation.

A water cooling solution removes heat from the motor more efficiently and the water’s high heat capacity leads to homogenous temperatures in the whole spindle. It dampenes vibrations and is the quietest cooling solution of all. On the other hand, the system is much more expensive, more complex, takes more space for pump, radiator and equalizing tank etc., and requires maintenance.

The planning phase

After studying the manuals of all the parts I had ordered, I created a schematic circuit diagram for the spindle subsystem.

At the top you see the VFD with its terminals. X1 is the mains power input with disconnector switch S2. X2 connects spindle to the VFD and X3 is the signal port to the CNC. In the lower right, there’s a DC power supply T1 that feeds the forced-air cooling and provides a logic voltage rail. The spindle M1 motor’s temperature is monitored by the thermistor R3 (where I forgot to add the label.) The spindle cooling motor M2 is controlled by a time relay which is in turn controlled through the RUN signal. A digital multi-purpose output is connected to a red LED D1 that is configured to light up if an overload warning is present.

The circuit plan remains pretty straight-forward. So i did not invest time into neat drawings. To make it look even more authentic, I spilled some ☕ on the paper.

This is the pinout of the spindle connector X2 and its corresponding terminal on the VFD. It was useful several times when adding connectors to the shielded motor cable (orange in color, ref. to banner image at the top), and during unit testing.

I noted down the fastening torque requirements for each screw including the terminals and prepared the tools I’d require to both prepare the switchbox’s in- and outlets and air vents, as well as the mount and wiring setup.

The construction phase

Spindle VFD Cabinet build↗

Commisioning

Before taking the system into productive use, I ran the test plan. When it was complete, I executed a Spindle grease distribution run according to the manufacturer’s instructions. Then, I calibrated the 0-10V analog out in my CNC software to match VFD frequency / spindle RPM. Example: S10000 should generate 3.33V at the output and make the VFD run at a frequency of around 175Hz to account for slip.

Finally, I was able to run some light jobs to see that the system works as intended: 8mm PMMA, full slot, single pass in Z+=8mm, 4mm single flute (polished) carbide cutter at S=26kRPM and F=3000mmm/min

The analysis phase

This phase will follow later, maybe a year into working with the new spindle. I’ll have a look at the following:

Was the investment worth it (roughly 8x more expensive than a router motor)?
Did I use the additional capabilities (higher power & more RPM)?
Does it produce noticeably better quality results?
Is it more reliable and requires less maintenance to my previous motor?

But what I already now can say: I learned a lot, it was fun designing, wiring, setting up the system, and exciting to actually see it worked out as I intended 🙂

Spirograph

Schallbert — Mon, 01 Jan 0001

The spirograph

Spirographs are mathematical toys. With their help it is easy to draw repetitive, curved shapes. The simplest version is made up of a cogwheel that runs on the inside of a static ring gear. The running wheel has a couple of holes to take the tip of a pen. The holes are located out of center of that cogwheel so, when rotating the wheel inside the hollow one, it will create parabolic shapes.

The number of rotations needed to draw a closed shape depends on the relation of tooth count between the two wheels.

Version 0

Our initial design was created together with the guys from CADasCAM in the workshop. It is a 14:9 tooth combination, so the lowest common multiple (aka LCM) is 126 which would create tight drawings with 126 / 9 = 14 corners. We had it manufactured in Aluminium and discussed about a good milling strategy using the CAM part of their software.

Your browser does not support the video tag.

Milling Spirograph_V0 on an overclocked CompactLine CNC

The parts turned out good and the wheels would allow cog to cog placement but unfortunately, it wouldn’t turn because the inner wheel’s next cog wouldn’t hit the flank but instead the ring gear tooth’s head.

Version 1

When the workshop was over, I took the design home and tried to improve. I shortened the tooth heads of the outer wheel to fix the previous issue and had some rounds added so the wheels wouldn’t feel edgy.

I had the design manufactured in 18mm Beech glued wood. I introduced two-sided milling here because chamfering by hand turned out to be a bit tedious at these more complex shapes. For that, I did not cut right through the material but left a 1mm “onion skin” at the bottom which would hold everything in place.

When I finally had the edges of the lower side chamfered, the bit would pierce the onion skin and practically cut out the workpieces so that I’d just have to snatch them off the machine bed. I just had to sand down the burrs which took less than five minutes of manual rework.

But it still wouldn’t fit. This time, because the flanks of the inner wheel were too wide. As a consequence, they would not allow full engage depth which in turn makes the next tooth of the inner wheel hit the tooth’s head of the ring gear.

Version 2

I figured out that in Version_1, the inner wheel had a positive tooth profile shift value while the ring wheel was left at zero. This couldn’t work at all so I fixed it. I also reworked both wheel’s concave radiuses so a 6mm endmill would be able to properly carve all details. Finally, I added some play for the teeth flanks and allowed for more room in the teeth’s troughs for better tooth meshing.

To give more room to the inner wheel, I mutated the gear combination to 16/7 that will draw 112 / 7 = 16 corners (even tighter drawing).

This time I had it manufactured in 5mm extruded Acrylic (PMMA), opaque white. I went with a similar two-sided milling strategy as discussed above, but I added a finishing run for superior quality of cut. I used the following parameters:

Cutter, Material: PMMA (Acrylic)	Work type	Tooth	Dia [mm]	Speed [RPM]	Feed [mm/min]	Z+ [mm]	XY+ [mm]
carbide hawk beak upcut FSECO1	Roughing	1	4	24000	3000	8	3.6
carbide hawk beak upcut FSAC	Finishing	1	4	26000	3000	8	0.2
carbide taper 90° FEF	Chamfers	3	10	15000	5000	3	3

And it worked flawlessly 😄, quality of cut was very high and all I did manually was to use a scraper for deburring that took under a minute.

Your browser does not support the video tag.

Updated tooth parameters allow smooth meshing

But still there was room to improve:

Holes right in the teeth don’t make sense because the inner wheel would not want to turn when engaged
5mm holes are a bit too small to home all sorts of crayons
16 teeth modul10 are not enough to fill a DIN A4 sheet of paper
The elliptic shape looks nice but is a bit hard to keep in place

That’s why I made one more step in the evolution:

Version 3

I made another slight update over Version_2, this time with an 18/10 pairing (LCM of 90, so 9 corners). I reworked the hole distances in form of a fibonacci spiral to yield more expressive designs. Finally, I updated the outline to match DIN A4 paper size so that the designs now would be centered.

I used the same parameters as in the previous version for manufacturing. See for yourself how good it came out. I wonder if I can get the last 5% out of quality by avoiding burrs altogether. Alone, I wouldn’t knnow how. Looks like the burr is created when the tapered cutter pierces the underside of the material to make the final cutout on my vacuum table.

Another thing I noticed is that the machine creates chatter marks when accelerating to target speed in G01 operations. I guess this is due to the limited frame stiffness. It is noticeable only under a few angles but not palpable.

The finished product then looks like this. Due to the chamfers and rounded edges, it feels really smooth. The gears run effortlessly the 18/10 cog combination has so many teeth meshed at all times that even kids aged 5 have no issue to produce perfect shapes (tested it).

And finally, I’m also happy with how the drawings look. Bold, big enough (15x15cm / 6" by 6"). Lines stand together not too tight. The higher number of holes that the larger inner wheel allows help generating higher diversity figures as well.

I will donate the specimen to a nearby school. Let’s see how they use them.

You want to DIY?

No problem. Download a ZIP package here.

What’s in the package?

CADasCAM project that allows all sorts of wheel pairing configurations
Drawing files (DXF) both for front and rear milling
EstlCAM project files along with my tool database
CNC operations files (tap) that fit my machine
A readme (txt) that defines XY zero and required offsets

Does it cost anything?

No, it’s for free. Note that the License I use makes the package “free software”. Please contact me if you want to utilize anything of the designs in other contexts than home and private usage.

Disclaimer

As usual, the files are available for download free of charge. In turn, I do not take any responsibility or liability for their contents.

WARNING ⚠️

These files provide or allow output of G-code instructions for real machinery that does actually do things in the physical world. There might be code errors or bugs that could potentially lead to machine crashes or even worse. I do not take liability for work accidents, system failures, equipment breakdown, loss of production, flow disturbances, or other negative effects that may be caused through the files I provided.

NOTE ℹ️

It is the machine operator’s responsibility to carefully review G-code and to make sure that it works as intended on the specific machine without causing any harm.

This site Demo

Schallbert — Mon, 01 Jan 0001

Project stats

Difficulty: medium 3/5
Cost: 0€
Time: ~15h

The idea

The idea originated through my MobFobAmp project of which I didn’t have a github repository as there was no code or similar documentation to keep. So I decided I wanted a website so I could host instructions for this Open Hardware project for others to build their own copy. I wanted an easy-to-host solution that wouldn’t take too much work to set up. As I don’t have an own server, I asked for help. moritzmar recommended to use github pages as a site host to start with for easy auto-deployment and global scope. So I gave it a try.

Getting started

I figured out that a static site would be the right thing to use, without the need for a database or huge Content Management frameworks as my site layout should be simple and minimalistic; I just wanted to host this one project in the beginning. Github recommended to use Jekyll

Jekyll Logo ^[1]

as a site generator. I had some trouble with the Windows Subsystem for Linux (WSL) as it wouldn’t accept my terminal inputs at some point, so I followed the guide and used the RubyInstaller which worked right out of the box for me.

Site setup

I realized that I might want to document more of my projects, and not only the “hardware” ones. That’s why I came up with the idea of grouping projects together, each with a header, a little teaser text, and a thumbnail image. Clicking on which would lead to the individual project page. Which would involve a lot of programming for page setup and layouting which I didn’t want to spend too much time on - my other projects were waiting…

First try with Themes

So I followed another suggestion by Github pages: The usage of pre-defined “Themes”. I first used the Agency Theme. This one has a great landing page with a huge picture and it supports collections that enable the group-the-project thing that I wanted. Unfortunately, the theme is a “remote theme” so customization was limited, e.g. I didn’t manage to have a colletion point to another page, it would always open a “popup” and I just didn’t want this. (Later, I figured out that remote layouts can be overridden, but that was too late so I already switched to another theme supplier…) The theme I’m currently using the Minimal Mistakes Theme (see site footer) which is very well documented. After downloading and a couple of first steps, I struggled with custom colors (_sass) and the collections - again - as I didn’t understand at first how Jenkill would do the bindings behind the scenes. Some of my struggles can be found in the Blog posts from June 2021.

Site layouts: hero screens

A bit of thinking should go into how the site layout should be like, to make navigation easy and intuitive. The approach will differ with every page type due to differences in complexity and document structure.

Take the landing / home page for example, I want the navigation to be sticky so that the site’s categories are always visible. It shall receive a huge, wide overlay image with the page description. Below, latest posts will be listed. This is the example for the collection’s page hero screen:

The projects and their contents shall contain a page navigation in the left sidebar, may contain a scrollable header image, and has a wide setting to display a high amount of text in the content area.

The “hero screen” scetches for this site’s layouts can be found here.

Collections

Collections are a great way for grouping content. Look at my Open Hardware Projects page. You can find all projects in this category with a little teaser image and description here to get an overview.

This is how the collection’s configuration can work out:

Think of a fitting name for your collection, e.g. electronics.
Collection name is defined in _config.yml as collections: electronics.
Collection name has to be set in #defaults as type: electronics so that Jekyll knows how to interpret the actual content.
Collection page has to be created (e.g. in the _pages folder) with the attribute collection: electronics. This is the page that will actually display the overview of items in the collection.
Content folder with the same name as the collection’s name with leading underscore has to be created: _electronics.
All documents in this content folder will show up in the collection page automatically after a full build.
These documents should get a “front matter” (section before the actual content in the file, marked with ---) so that they can be displayed as described in the corresponding _layouts file, e.g. collection

More than the standard Theme

I have overridden some theme default and I’m using most of the existing Theme infrastructure. But my site has a couple of special needs so I had to customize and add Theme features. These are described in the following sections.

As my site navigation tree is pretty flat and I don’t have lots of pages to navigate to, I wanted the left sidebar to hold the page navigation rather than the site navigation. Thus, I wouldn’t need a right sidebar at all, so that there would be more space for the page content.

Table Of Contents on the left

As written here and there, I never was 100% happy with the way my project pages looked like:

The original idea of the theme is that the content is framed by two sidebars: The left one can either display Author information, custom content, or the site navigation. The content area is configurable between wide and normal, in the latter case leaving space for a second sidebar on the right. The right sidebar is configurable to not be there at all or to show the page’s table of contents.

So the target I set was to eliminate the right sidebar for my design, and instead of having the site navigation in the left sidebar, move the page navigation there.

It’s all about the /_layouts folder that keeps information about how a page layout should look like. The page design in question is called single.html and it at some point calls include sidebar.html which resides in /_includes/sidebar.html. So I thought of a new value I could use to control showing the table of contents within the left sidebar, chose toc_left and added a condition:

<div class="sidebar sticky">
<!-- ...
{.% if page.toc_left %}
 {.% include toc_left nav=page.toc_left %}
{.% endif %}
... -->
</div>

(By the way: I had to comment-out and add .’s to the jinja code, Liquid would otherwise interpret these as commands!)

Implementing the Table Of Contents

I wanted it to show up in the left sidebar container. So I just had to create a toc_left file that would call the included toc generator like this:

<aside class="toc_left">
<nav class="toc" markdown="1">
<header><h4 class="nav__title"><i class="fas fa-{{ include.icon | default: 'file-alt' }}"></i>
<!-- ...
{.{ include.title | .default: site.data.ui-text[site.locale].toc_label }}</h4></header>
{.% include toc.html sanitize=true html=content h_min=1 h_max=6 class="toc__menu" skip_no_ids=true %} 
... -->
</nav>
</aside>

Audio Embeds

Audio embeds are these little player-like things on a website, often forwarding to a streaming platform or similar. I need them for some of my audio projects like this mobfobamp: just a caption and the player below.

This post helped me a lot, so I could just add it to the _include, modify it a bit, and it worked!

Placing the favicon

I followed this guide to place a favicon for the site and it worked flawlessly.

The favicon’s path has to be valid independently of the folder the rendered page resides in. I had to add ../ according to the pages’ depth within the site’s folder structure to make sure that the favicon could be displayed not only on the “home” page, but also from sub-paSges.

Optimizing for search engines

As my site was not found by search engines for a couple of weeks, I decided to perform so-called “SEO” (search engine optimization). That’s why I added jekyll-seo-tag and e.g. reworked all my links following Juliette Sinibardy’s SEO 101. When complete, I put my webpage through Lighthouse’s web-analysis audit.

For a first try, I think it looks pretty well. Biggest issue on the performance side seems to be CSS overhead.

Content Security Policy

The Lighthouse audit revealed that it would be a good idea to restrict external scripting to my site, so I introduced a Content Security Policy (CSP) like this:

webrick:
 headers:
 Content-Security-Policy:
 default-src 'self'
 script-src 'self';
 connect-src 'self';
 img-src 'self';
 style-src 'self';
 font-src 'self';

Initially, unfortunately, my home page’s backup image wouldn’t show, my embed-audio files wouldn’t play and all icons loaded externally were gone as well. Adding media-src: 'self'; and some more reading in Jakob Wilforss’ blog and especially Lighthouse’s recommendation helped to solve this. By using the Browser’s Web developer Tools, I figured out the missing resources the theme is loading in the background, so my updated CSP looks like this:

webrick:
 headers:
 Content-Security-Policy:
 default-src 'self' 'unsafe-inline' https://cdn.jsdelivr.net/; 
 script-src 'self';
 connect-src 'self';
 img-src 'self' 'unsafe-inline' https://upload.wikimedia.org/wikipedia/commons/ https://raw.githubusercontent.com;
 style-src 'self' 'unsafe-inline' https://cdn.jsdelivr.net/;
 font-src 'self' 'unsafe-inline' https://cdn.jsdelivr.net/; 
 media-src 'self'; 
 object-src 'none';
 base-uri 'none';

Errors

I’m collecting some simple errors here that cost me quite some time fixing.

Jekyll serve: `UTF-8` incompatible character encoding

You enter bundle exec jekyll serve and get a strange UTF-8 “incompatible character encoding” error? The cause is simple: One of your files most likely has no header that can be converted into the front matter of the page that is generated from that file. Just add your usual header to your file like so and it will just work:

---
title: "Feeds and Speeds database"
description: "for (light) CNC milling machines"
header:
 image: "blah/testimage.jpg"
[...]
---

Webpage is not being displayed at all: `404`

This error was very, very hidden. I found it by accident browsing my site online only, as localhost serve was not affected. The symptom is a no-show blank page.

Cause: Due to translation, my paths are getting longer and I wanted subpages to not have extremely long links. That’s why I was using the permalink: Attribute of the pages’ fontmatter. Now, only if you have a “trailing slash” here, the page will display on the remote server.

# Wrong:
---
title: "This Page won't be displayed"
permalink: /projects/pageerror
# [...]
---

# Correct:
---
title: "This Page will be displayed!"
permalink: /projects/nopageerror/
# [...]
# Unfolds on the english blog to:
# /en/projects/nopageerror/
---

Site hosting

This site itself is a normal Gitea repository. It is built automatically with Gitea Actions once changes are added to the repository’s main branch.

Summary

To get familiar with how Jekyll creates the site, how the theme would interact with my commands and how to override Theme behavior cost quite a couple of hours, but I think this site has a clear structure and a more or less “professional” look - And I didn’t have to write a single line of JavaScript or HTML to get this done. Nice!

Contents written in markdown *.md
Basic site configuration in YAML _config.yml
Page layouts written in html and jinja *.html, placed in _layouts and _includes
Page variables and appearance in SASS *.scss, automatically generating css files

Contribute?

You have ideas, criticism, or interesting stuff you’d like to let me know? Get started on my Discussions page!

Tonuino - alternative Firmware

Schallbert — Mon, 01 Jan 0001

Project stats

Difficulty: hard 5/5
Cost: 0€
Time: ~400h

Impressions

My alternative tonuino-Firmware in action.

NFC-Tags are birds that can be placed in the bird's house to play an album.

A microcontroller implements my encoder library to run this buttonless design of Tonuino.

Tonuino in an old battery-powered guitar amp.

The player is controlled via three buttons that I took from an old computer keyboard.

A software written for the Arduino Framework that controls an Mp3 Player (with SD card), an NFC Tag Reader, and provides user controls that together work like a jukebox that can be controlled via NFC Tag. For example, an NFC tag can be linked to a folder on the SD card containing audio files. The tag will also store information about the requested play mode for this album, e.g. “Random”. Once a linked tag is placed on this jukebox, the configured folder will play with the selected playmode. User controls allow to select the track, change volume and even to navigate a voice menu that allows deleting a tag, linking/ configuring a tag, and to lock/ unlock the input keys if required.

Motivation

Idea and fantastic execution found on Thorsten Voß’s blog. It began with a feature (encoder support) that I wanted to add. Unfortunately, the original code was a single-file application with several thousand lines of code and poor readabilty, so I had a hard time getting my changes in. So I re-wrote the code from scratch for better structure, readability, maintainability, and extensibility. By doing this, in parallel I read some books about Object Oriented design in C++, Clean Code and Software Design Patterns and applied some where I tought they’d fit in. This way, I was able to both improve modularity and readability of the code, plus I taught myself how to write code in a “bigger” project.

Hurdles to overcome

This was first real OO C++ project, and, maybe, it was a little big to start with. All the new-to-me concepts, writing to an interface, first-time usage of Platformio as IDE, utilizing the googletest unit test framework, using coding patterns like factory or Dependency Injection took nearly a year of evenings before this project could be completed. And there’s always something left to refactor and to improve.

Quick facts

Unit test suite of 250+ test cases
Serial debug output configurable
Loosely coupled C++ OO architecture
Individual modules with clear tasks, scaleable and easily accessible for future feature add
custom Hardware abstraction layer; most of the code should be portable to other mcus without changes
custom Dependency Injection framework (Loader class)

Features

Configurable user input (buttons or encoder)
Auto-Poweroff when no button pressed for a configurable time
Power on via “play” button press
Autoplay feature
Status Led feature
User Input lock/unlock feature
Multiple playmodes [Album, Random, SaveProgress, OneTrack] available per Nfc Tag
Voice menus for linking or deletion of Nfc Tags
Optimized for battery applications (e.g. powerbank) using sleep states
Low power consumption @5V: ~40mA in idle, ~75mA playing medium volume
Config file for init volume, lullabye timer etc.
Auto-recovery from stuck prompts
Voice prompts for most common state errors

Features not incliuded

No powerbank state of charge detection
No config settings menu (init volume, lullabye timer duration, standby duration etc.)
…

Documentation

The Project is build for the Arduino Framework - tested on an Arduino nano board - using the PlatformIO IDE. The following sections show the design.

Project Module overview

Folder name in `/lib`	Purpose
Arduino	minimalistic close-to-Hardware implementations, not unit-testable
Arduino_HardwareAbstraction	Hardware abstraction to enable portability and testing
Config	System configuration parameters
Folder	Playlist and playmode business logic
Loader	Dependency Injection Framework
MessageHandler	System messages and Debug Framework
Mp3	Mp3 control (Status, Folder, Prompts, Advertisements)
Nfc	Tag control (Status, Read, Write, Delete)
PowerManager	Control Status Led and Keep Alive based on system state
Tonuino	Main task scheduler
UserInput	Button or Encoder input handling
Utilities	Timers, Led Control, Pin control
VoiceMenu	Link / Delete / Config menu business logic

External Libraries

Auto-install through Platformio on initial build through Library Dependency Finder.

Class Diagrams

Class diagrams would have been too much non-automated work. Take this instead, please. Software module overview It should give an accurate understanding of how the software module are interacting and what APIs the modules are offering.

Get started!

It’s DIY time! Clone my repository to get started with. Read the README.md document. Build it with PlatformIO. Below you can find what you need to get started with your unique speaker design.

Testing

Unit tests

Unit tests are written with the gtest C++ unit test framework. The reside in the test/desktop folder. Note that googletest will require gcc with some libraries to be installed, how to can be found in one of my other projects. Once these prerequisites are complete and a simple ASSERT_TRUE(false); test fails as expected using the gtest environment, the project’s Unit Tests can be built and run using pio test -e desktop -f desktop, detailed out here, in the PlatformIO CLI (terminal).

Acceptance tests

Although these tests could be automated, it is much easier to just perform those tests by hand after you programmed your µC and put all electronic components together. Each line in the tables below is a test case on its own. The test suites (Heading name) do have a System Pre: property, that needs to be re-established prior to executing the individual test cases. If the expectation clause is met, the test is a PASS.

Switching ON

System Pre: System is OFF

Action	Expectation
press Play/Pause button	LED flashes slowly?
press Play/pause button	Welcome prompt plays?

Switching OFF

System Pre: System is ON

Action	Expectation
Playback on pause, no button input	System switches off after a time?
System shutdown	Farewell prompt plays?
System shutdown	LED switched off?

Play Help Prompt

System Pre: System is ON

Action	Expectation
Long press Play/Pause button	Help prompt plays?
Help prompty playing	Can be interrupted with any button press?

Behavior without Tag

System Pre: System is ON, no Tag present

Action	Expectation
press Play/Pause button	prompts “couldn’t find track” error?
press Next button	prompts “couldn’t find track” error?
press Prev button	prompts “couldn’t find track” error?
doubleclick Play/Pause button	Delete Menu prompt played?

Behavior with Tag

System Pre: System is ON, linked Tag available

Precondition	Action	Expectation
no Tag placed	place known Tag	starts Playback of correct Folder?
active playback	press Next button	plays next track?
active playback	press Next button	next track in accordance with selected playMode of Folder?
active playback	long press Next button	increases volume?
active playback	press Prev button	plays previous track?
active playback	press Prev button	previous track in accordance with selected playMode of Folder?
active playback	long press Prev button	decreases volume?
active playback	press Play/Pause button	pauses track?
paused playback	press Play/Pause button	resumes track?
active playback	doubleclick Play/Pause button	Lock prompt played?
active playback	doubleclick Play/Pause button	Playback resumes after Lock prompt played?
active playback	doubleclick Play/Pause button	locks button input?
locked button input	doubleclick Play/Pause button	Unlock prompt played?
locked button input	doubleclick Play/Pause button	Playback resumes after Unlock prompt played?
locked button input	doubleclick Play/Pause button	unlocks button input?
paused playback	doubleclick Play/Pause button	Playback resumes after Lock prompt played?

Behavior with unlinked Tag

System Pre: System is ON

Action	Expectation
place unlinked Tag	plays LinkMenu Prompt?
navigate through LinkMenu	plays Configuration Success Prompt?
place Tag again	plays linked folder?

Hardware

Material list

Minimal discrete parts approach

Amount	Item	Purpose
1	Arduino (e.g. Nano)	Brains
1	Dfplayer Mini Mp3 Player	Mouth
1	Micro SD Card	Memory
1	MFRC522 Nfc Reader	Psi-Sense
1	Bistable Relay 5V, e.g. HFE20	Coffee - no sleep
2	Diodes e.g. 1n4007	vene valve
1	PNP Transistor e.g. BC327	Coffee maker switch
2	Resistors 1k	blood-stream conditioner
1	Resistor 220	dim blink strength
1	LED, color of choice	blink
1	Powerbank/ 5V supply	Food
3	push buttons, e.g. cherry MX keys	pressure sensors
1	speaker, e.g.5W@4Ohms	Lungs
1	USB-A plug	Straw
1	housing	Body

Additional coponents and tips

Jumper wires, PCB, Sockets, and expendable materials as you deem fit Same as for the “original” project; Alternatively buy one rotary encoder instead of push buttons and configure the project to use it. As the system is optimized for battery usage, take a bi-stable relay or a JFET transistor (with low gate voltage and low Source-Drain voltage drop) for keepAlive functionality mitigating additional current add through always-on relay coils. A cheap powerbank will last for many hours. Alternatively, e.g. 3xAA batteries can be used - but the Dfmini will really make some noise if not enough current can be supplied due to running resets (and may even get bricked through current surges, so beware).

Parameter	Value
Pull-out parameter \( f_{ax,k} \)	14 N/mm²
Torsional strength \( f_{tor,k} \)	3 Nm
Screw-in torque \( R_{tor,mean} \)	\( \frac{f_{tor,k}}{R_{tor,mean}} \) ≥ 1.5
Pre-drilling in softwood	2.5 mm
Pre-drilling in hardwood	3 mm

Schallbert's Blog

Hardware Security Key: add SSH

Motivation

Setting up SSH access for hardware security keys

What doesn’t work

Meaning of the command and the most important options

What was the problem?

What does work, however

What verify-required requires

Installing ssh-askpass

Annoying: Gnome pop-up ‘Allow inhibiting shortcuts’

Transferring to Windows systems

VPS: Be careful when replacing keys

Manually store new keys on the VPS

Was the key transferred successfully?

How to avoid ssh-copy-id

Thoughts on convenience and usability

Hardware Security Keys

Definitions: TPM and HSK

What happened so far

Objectives

Yubikey

Yubico Authenticator

Concept of the HSK

Spare key

NFC

What I find myself missing when using the Yubikey

Touch, password, PIN, PUK, management key, passphrase

Feature set and cryptographic landscape

Two-factor authentication: “Yubikey: Accounts”

Passkeys ‘Yubikey: Passkeys’

Keys & Certificates ‘Yubikey: Certificates’

Public/Private Key Pair

Certificates / PIV

act_runner rootless: no start

The error

Issue ticket on Gitea

Problem with kernel permissions?

Why containers are not allowed to start other containers

Consequences if something goes wrong despite DinD

Simple solution: Start the DinD container as privileged

Dead end: Only grant the rights that are absolutely necessary

Future solution: Virtualisation or daemonless

Linux Security Modules (seccomp, AppArmor, SELinux)

AppArmor

Simple solution: Disable completely

The correct solution: Tailor settings for each application

Engrave Stainless Steel A2 / AISI304

Why this attempt?

The Material

The Tool

Discussion about milling parameters

Machine Kinematics

Preparing the Engraving Process

Procedure

Result

Automate tool length measurement

Question: How does the macro calculate the tool length?

Answer:

Question: How do I calculate the workpiece height after a tool measurement?

Answer: Several solutions

Solution #3: Shift the coordinate system

Offset of workpiece zero vs. workpiece coordinate system

Prerequisites

Steel sheet (SECC) milling

The Project

The Challenge

The Specifications

Tool 1: 3mm Drill Bit

Drilling Parameters

Drilling Preparation

The Result

Problem Analysis

The Solution

Tool 2: 6mm Solid Carbide End Mill

Cut Quality

Jekyll-dockerimage: Bundler and Gemfile

What is bundler?

bundler in the CI pipeline

No write permissions

What `verify-required` requires

Simple solution: Start the DinD container as `privileged`

Optimize `Gemfile.lock` for different environments

Solution via `banaction = iptables[type=multiport]`

Rootless act_runner for Gitea

Danger from `docker.sock`

Do not include the Docker socket `/var/run/docker.sock` in other containers

Directory `filter.d`

`jail.local` file