removing open ports when sibling project removed

README.md

@@ -10,7 +10,7 @@ wget -qO- https://gitea.young.computer/david/MiniServer/raw/branch/main/install.
 update
 ```
 
-This pulls the latest MiniServer into `~/.local/share/MiniServer`, then runs each step in `updates.d/` in order: the Arch keyring and system packages (plus any packages newly added to `packages.conf`), AUR packages, pm2, Cockpit, the firewall (services and ports from `firewall.conf`), sibling projects, cache cleanup, and a reboot check. If a step fails, the others still run and the failed ones are listed at the end.
+This pulls the latest MiniServer into `~/.local/share/MiniServer`, then runs each step in `updates.d/` in order: the Arch keyring and system packages (plus any packages newly added to `packages.conf`), AUR packages, pm2, Cockpit, sibling projects, the firewall, cache cleanup, and a reboot check. If a step fails, the others still run and the failed ones are listed at the end.
 
 ## Refresh
 
@@ -33,3 +33,7 @@ It asks for the repository's git URL, clones it into `~/.local/share/siblings/`,
 Use `sibling list` to see what's installed and `sibling remove {name}` to uninstall one.
 
 To make your own repository work as a sibling project, see [skills/sibling-project-format.md](skills/sibling-project-format.md).
+
+## Firewall
+
+firewalld blocks incoming connections except for the services and ports listed in [firewall.conf](firewall.conf) and in each sibling project's own `firewall.conf`. Each `update` opens anything newly listed, and closes ports MiniServer opened that nothing lists any more, for example after `sibling remove`. Rules you add yourself with `firewall-cmd` are left alone.

firewall.conf

@@ -1,5 +1,7 @@
-# Allowed through firewalld (default zone) by install and update.
+# Allowed through firewalld (default zone) by install, update and sibling.
-# Removing an entry here does not close it; use firewall-cmd for that.
+# Sibling projects can have a firewall.conf in this same format at the top of
+# their repo. Removing an entry closes it on the next update, unless another
+# firewall.conf still lists it or it was open before MiniServer opened it.
 
 # firewalld service names (see `firewall-cmd --get-services`)
 FIREWALL_SERVICES=(
@@ -12,4 +14,7 @@ FIREWALL_SERVICES=(
 # Extra ports, e.g. for pm2 apps: "3000/tcp"
 # Docker containers' published ports don't need to be listed here.
 FIREWALL_PORTS=(
+  3000/tcp
+  8080/tcp
+  8081/tcp
 )

install.sh

@@ -137,7 +137,7 @@ setup_cockpit
 echo "Cockpit is at https://$(uname -n).local:9090"
 
 # Set up firewalld (installed with the utilities above)
-setup_firewall
+setup_firewall || echo "Some firewall rules could not be applied, see above. Run 'update' to retry."
 
 # Install Sway config files
 if [ ! -d "$HOME/.config/sway" ]; then

sibling

@@ -5,10 +5,14 @@
 # A sibling is a git repository cloned into ~/.local/share/siblings/<name>.
 # If it has an install.sh, that runs once when the sibling is added.
 # Every `update` pulls each sibling and runs its update.sh, if it has one
-# (see updates.d/71-siblings.sh).
+# (see updates.d/71-siblings.sh). Ports listed in a sibling's firewall.conf
+# are opened when it's added and closed when it's removed.
 
 SIBLINGS="$HOME/.local/share/siblings"
 
+# Provides setup_firewall
+source "$HOME/.local/share/MiniServer/utils.sh"
+
 usage() {
   cat << EOF
 Usage: sibling [command]
@@ -69,6 +73,15 @@ add_sibling() {
     fi
   fi
 
+  # Open the ports listed in the sibling's firewall.conf
+  if [ -f "$dir/firewall.conf" ]; then
+    echo "Applying $name's firewall.conf..."
+    if ! setup_firewall; then
+      echo "Some firewall rules could not be applied, see above. Fix the problem and run 'sibling add $url' again."
+      return 1
+    fi
+  fi
+
   # Confirm the install
   if [ -d "$dir/.git" ] && git -C "$dir" rev-parse --verify --quiet HEAD > /dev/null; then
     echo "Installed $name at $dir ($(git -C "$dir" log -1 --format='%h %s'))."
@@ -105,6 +118,8 @@ remove_sibling() {
   case "$response" in
     [yY][eE][sS]|[yY])
       rm -rf "${SIBLINGS:?}/$name"
+      # Close the ports only this sibling's firewall.conf asked for
+      setup_firewall || echo "Some firewall rules could not be updated, see above. 'update' will retry."
       echo "Removed $name. Anything its install.sh set up outside that folder is still there."
       ;;
     *)

skills/sibling-project-format.md

@@ -8,13 +8,14 @@ one with MiniServer's `sibling` command, and from then on MiniServer's
 
 Follow this document when you are working in a sibling repository and need it
 to install and update correctly under MiniServer. A repository needs no
-special files to be installable. It becomes useful by adding two optional
+special files to be installable. It becomes useful by adding these optional
-scripts at its top level:
+files at its top level:
 
-| File         | When it runs                                                 |
+| File            | When it's used                                               |
-|--------------|--------------------------------------------------------------|
+|-----------------|--------------------------------------------------------------|
-| `install.sh` | Once, when the user runs `sibling add`, and again if they re-run it |
+| `install.sh`    | Runs once, when the user runs `sibling add`, and again if they re-run it |
-| `update.sh`  | On every `update`, right after the repo has been pulled      |
+| `update.sh`     | Runs on every `update`, right after the repo has been pulled |
+| `firewall.conf` | Read on `sibling add`, `sibling remove` and every `update` to open the project's ports |
 
 Don't also copy a hook into MiniServer's `updates.d/` for the same project, or
 it updates twice.
@@ -29,7 +30,9 @@ it updates twice.
    already holds a clone of the same URL, runs `git pull --ff-only` there.
 3. If `install.sh` exists, runs `bash install.sh` with the repo folder as the
    working directory. Its input and output are the user's terminal.
-4. Reports success only if every step above succeeded.
+4. If `firewall.conf` exists, opens the ports it lists (see
+   [Firewall](#firewall)).
+5. Reports success only if every step above succeeded.
 
 ### `update`
 
@@ -42,13 +45,14 @@ Next, `updates.d/71-siblings.sh` takes each sibling in alphabetical order and:
 
 A failed pull or a non-zero exit from `update.sh` is printed and listed in
 `update`'s "Done, with errors in: …" summary. Other siblings still update.
-MiniServer's end-of-run hooks (cleanup, reboot check) run after all
+MiniServer's end-of-run hooks (firewall, cleanup, reboot check) run after all
-siblings.
+siblings, so a port added in the latest commit opens in the same update.
 
 ### `sibling remove <name>`
 
-This only deletes `~/.local/share/siblings/<name>`. MiniServer has no uninstall
+This deletes `~/.local/share/siblings/<name>` and closes the ports that only
-step for siblings, so anything `install.sh` set up elsewhere stays.
+its `firewall.conf` asked for. MiniServer has no other uninstall step for
+siblings, so anything `install.sh` set up elsewhere stays.
 
 ## Rules for the Repository
 
@@ -110,6 +114,34 @@ step for siblings, so anything `install.sh` set up elsewhere stays.
 - It's fine to omit `update.sh` if pulling the repo is enough, for example when
   `~/.local/bin` holds symlinks into the repo.
 
+### Firewall
+
+MiniServer runs firewalld, which blocks incoming connections except for the
+services it allows (SSH, mDNS, Cockpit and a few ports). If the project serves
+anything other machines connect to, list its ports in `firewall.conf` at the
+top of the repo:
+
+```bash
+# Ports other machines need to reach. Format: <port>/<protocol> or
+# <first>-<last>/<protocol>, protocol tcp, udp, sctp or dccp.
+FIREWALL_PORTS=(
+  5000/tcp
+)
+
+# Optional: firewalld service names (see `firewall-cmd --get-services`)
+FIREWALL_SERVICES=(
+)
+```
+
+- **Don't open ports from `install.sh` or `update.sh`** with `firewall-cmd`.
+  MiniServer can't close those on `sibling remove`.
+- **Only assign the two arrays.** The file is sourced by bash, but it's read on
+  every `update` and `sibling` command, so it must not do anything else.
+- **Removing an entry closes that port** on the next `update`, unless another
+  project still lists it or it was already open before MiniServer opened it.
+- Containers' published ports with Docker don't need listing: Docker opens
+  them itself.
+
 ### Keep pulls working
 
 `update` pulls with `--ff-only` and never merges or resets. A pull fails if:
@@ -208,6 +240,8 @@ changing.
       and is safe to re-run
 - [ ] `update.sh` (optional) is at the top level, idempotent, doesn't prompt,
       and prints what it's doing
+- [ ] Every port other machines connect to is in `firewall.conf`, and no
+      script runs `firewall-cmd` itself
 - [ ] Scripts find their own files relative to the script and set
       `MINISERVER="$HOME/.local/share/MiniServer"` themselves
 - [ ] Nothing writes to tracked files in the clone, and the default branch is

updates.d/80-firewall.sh

@@ -1,5 +1,8 @@
 #!/bin/bash
 
+# Runs after 71-siblings.sh so ports added in a sibling's latest commit open
+# in the same update
+
 source "$HOME/.local/share/MiniServer/utils.sh"
 
 # Defined after sourcing utils.sh, which has its own is_installed

utils.sh

@@ -83,16 +83,33 @@ setup_cockpit() {
   fi
 }
 
-# Allow the services and ports in firewall.conf through firewalld's default
+# Print the rules a firewall.conf asks for, one per line, as service:<name> or
-# zone, then make sure firewalld is running. Only adds rules, never removes
+# port:<port>/<protocol>. Sourced in a subshell so the file can't change
-# them, so rules added by hand or by sibling projects stay. Safe to re-run.
+# anything else.
+firewall_entries() {
+  (
+    FIREWALL_SERVICES=()
+    FIREWALL_PORTS=()
+    source "$1" > /dev/null || exit 1
+    for e in "${FIREWALL_SERVICES[@]}"; do echo "service:$e"; done
+    for e in "${FIREWALL_PORTS[@]}"; do echo "port:$e"; done
+  )
+}
+
+# Rules setup_firewall opened, so it knows which ones it may close again
+FIREWALL_STATE="$HOME/.local/state/miniserver/firewall-opened"
+
+# Allow what MiniServer's and each sibling project's firewall.conf ask for
+# through firewalld's default zone, and make sure firewalld is running.
+# Closes a rule only if setup_firewall opened it and no firewall.conf asks for
+# it any more (e.g. after `sibling remove`); rules that were already there or
+# were added by hand are left alone. Safe to re-run.
 setup_firewall() {
   command -v firewall-offline-cmd &> /dev/null || return 0
-  source "$HOME/.local/share/MiniServer/firewall.conf"
 
   # firewall-cmd only works while firewalld is running; before that, write the
   # permanent config directly so SSH is allowed before the firewall comes up
-  local fw running=0 changed=0 svc port
+  local fw running=0 changed=0 complete=1 status=0
   if systemctl is-active --quiet firewalld; then
     running=1
     fw=(sudo firewall-cmd --permanent)
@@ -100,39 +117,86 @@ setup_firewall() {
     fw=(sudo firewall-offline-cmd)
   fi
 
-  # Keep SSH reachable if sshd listens on a port other than 22
+  local wanted=() opened=() kept=() conf out entry kind value port
-  local ports=("${FIREWALL_PORTS[@]}")
+  for conf in "$HOME/.local/share/MiniServer/firewall.conf" "$HOME/.local/share/siblings/"*/firewall.conf; do
+    [ -f "$conf" ] || continue
+    if out="$(firewall_entries "$conf")"; then
+      [ -n "$out" ] && mapfile -t -O "${#wanted[@]}" wanted <<< "$out"
+    else
+      echo "Firewall: could not read $conf, so no rules will be closed this time."
+      complete=0
+      status=1
+    fi
+  done
+
+  # Keep SSH reachable if sshd listens on a port other than 22. If its config
+  # can't be read, don't close anything: that port could be the SSH port.
   if command -v sshd &> /dev/null; then
-    for port in $(sudo sshd -T 2> /dev/null | awk '$1 == "port" && $2 != 22 { print $2 }'); do
+    if out="$(sudo sshd -T 2> /dev/null)"; then
-      ports+=("$port/tcp")
+      for port in $(awk '$1 == "port" && $2 != 22 { print $2 }' <<< "$out"); do
-    done
+        wanted+=("port:$port/tcp")
+      done
+    else
+      complete=0
+    fi
   fi
 
-  for svc in "${FIREWALL_SERVICES[@]}"; do
+  [ -f "$FIREWALL_STATE" ] && mapfile -t opened < "$FIREWALL_STATE"
-    if ! "${fw[@]}" --query-service="$svc" &> /dev/null; then
+
-      echo "Firewall: allowing $svc"
+  for entry in "${wanted[@]}"; do
-      "${fw[@]}" --add-service="$svc"
+    kind="${entry%%:*}"
-      changed=1
+    value="${entry#*:}"
+    if ! { [[ "$kind" == service && "$value" =~ ^[A-Za-z0-9_.-]+$ ]] ||
+           [[ "$kind" == port && "$value" =~ ^[0-9]+(-[0-9]+)?/(tcp|udp|sctp|dccp)$ ]]; }; then
+      echo "Firewall: ignoring invalid $kind '$value'"
+      status=1
+      continue
+    fi
+    if ! "${fw[@]}" --query-"$kind"="$value" &> /dev/null; then
+      echo "Firewall: allowing $value"
+      if "${fw[@]}" --add-"$kind"="$value"; then
+        opened+=("$entry")
+        changed=1
+      else
+        status=1
+      fi
     fi
   done
-  for port in "${ports[@]}"; do
+
-    if ! "${fw[@]}" --query-port="$port" &> /dev/null; then
+  for entry in "${opened[@]}"; do
-      echo "Firewall: allowing $port"
+    [ -n "$entry" ] || continue
-      "${fw[@]}" --add-port="$port"
+    if (( ! complete )) || printf '%s\n' "${wanted[@]}" | grep -qxF -- "$entry"; then
-      changed=1
+      kept+=("$entry")
+      continue
+    fi
+    kind="${entry%%:*}"
+    value="${entry#*:}"
+    if "${fw[@]}" --query-"$kind"="$value" &> /dev/null; then
+      echo "Firewall: closing $value (nothing asks for it any more)"
+      if "${fw[@]}" --remove-"$kind"="$value"; then
+        changed=1
+      else
+        kept+=("$entry")
+        status=1
+      fi
     fi
   done
 
+  mkdir -p "$(dirname "$FIREWALL_STATE")"
+  printf '%s\n' "${kept[@]}" | sed '/^$/d' | sort -u > "$FIREWALL_STATE"
+
   if (( running )); then
     if (( changed )); then
-      sudo firewall-cmd --reload
+      sudo firewall-cmd --reload || status=1
     fi
   else
     echo "Starting firewalld..."
-    sudo systemctl enable --now firewalld
+    sudo systemctl enable --now firewalld || status=1
     # Docker sets up its firewalld zone only when it starts
     if systemctl is-active --quiet docker; then
       echo "Docker started before firewalld. Restart it (sudo systemctl restart docker) or reboot so its containers' ports work through the firewall."
     fi
   fi
+
+  return $status
 }