utils/lxc.sh
¶
With the use of Linux Containers (LXC) we can scale our tasks over a stack of
containers, what we call the: lxc suite. The SearXNG suite config is
loaded by default, every time you start the lxc.sh
script (you do not need
to care about).
Install LXD¶
Before you can start with containers, you need to install and initiate LXD once:
$ snap install lxd
$ lxd init --auto
To make use of the containers from the SearXNG suite, you have to build the LXC suite containers initial. But be warned, this might take some time:
$ sudo -H ./utils/lxc.sh build
A cup of coffee later, your LXC suite is build up and you can run whatever task you want / in a selected or even in all LXC suite containers.
Internet Connectivity & Docker¶
There is a conflict in the iptables
setup of Docker & LXC. If you have
docker installed, you may find that the internet connectivity of your LXD
containers no longer work.
Whenever docker is started (reboot) it sets the iptables policy for the
FORWARD
chain to DROP
[ref]:
$ sudo -H iptables-save | grep FORWARD
:FORWARD ACCEPT [7048:7851230]
:FORWARD DROP [7048:7851230]
A handy solution of this problem might be to reset the policy for the
FORWARD
chain after the network has been initialized. For this create a
file in the if-up
section of the network (/etc/network/if-up.d/iptable
)
and insert the following lines:
#!/bin/sh
iptables -F FORWARD
iptables -P FORWARD ACCEPT
Don’t forget to set the execution bit:
sudo chmod ugo+x /etc/network/if-up.d/iptable
Reboot your system and check the iptables rules:
$ sudo -H iptables-save | grep FORWARD
:FORWARD ACCEPT [7048:7851230]
:FORWARD ACCEPT [7048:7851230]
SearXNG LXC suite¶
The intention of the SearXNG LXC suite is to build up a suite of containers
for development tasks or buildhosts with a very
small set of simple commands. At the end of the --help
output the SearXNG
suite from the SearXNG suite config is introduced:
$ sudo -H ./utils/lxc.sh --help
...
LXC suite: searxng
Suite includes installation of SearXNG
images: ubu2004 ubu2204 fedora35 archlinux
containers: searxng-ubu2004 searxng-ubu2204 searxng-fedora35 searxng-archlinux
As shown above there are images and containers build up on this images. To show
more info about the containers in the SearXNG LXC suite call show suite
.
If this is the first time you make use of the SearXNG LXC suite, no containers
are installed and the output is:
$ sudo -H ./utils/lxc.sh show suite
LXC suite (searxng-*)
=====================
+------+-------+------+------+------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+------+-------+------+------+------+-----------+
WARN: container searxng-ubu2004 does not yet exists
WARN: container searxng-ubu2204 does not yet exists
WARN: container searxng-fedora35 does not yet exists
WARN: container searxng-archlinux does not yet exists
If you do not want to run a command or a build in all containers, you can build just one. Here by example in the container that is build upon the archlinux image:
$ sudo -H ./utils/lxc.sh build searxng-archlinux
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux pwd
Otherwise, to apply a command to all containers you can use:
$ sudo -H ./utils/lxc.sh build
$ sudo -H ./utils/lxc.sh cmd -- ls -la .
Running commands¶
Inside containers, you can run scripts from the DevOps tooling box or run what ever command you need. By example, to start a bash use:
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux bash
INFO: [searxng-archlinux] bash
[root@searxng-archlinux SearXNG]#
Good to know¶
Each container shares the root folder of the repository and the command
utils/lxc.sh cmd
handle relative path names transparent:
$ pwd
/share/SearXNG
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux pwd
INFO: [searxng-archlinux] pwd
/share/SearXNG
The path /share/SearXNG
will be different on your HOST system. The commands
in the container are executed by the root
inside of the container. Compare
output of:
$ ls -li Makefile
47712402 -rw-rw-r-- 1 markus markus 2923 Apr 19 13:52 Makefile
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux ls -li Makefile
INFO: [searxng-archlinux] ls -li Makefile
47712402 -rw-rw-r-- 1 root root 2923 Apr 19 11:52 Makefile
...
Since the path /share/SearXNG
of the HOST system is wrapped into the
container under the same name, the shown Makefile
(inode 47712402
) in
the output is always the identical /share/SearXNG/Makefile
from the HOST
system. In the example shown above the owner of the path in the container is
the root
user of the container (and the timezone in the container is
different to HOST system).
Install suite¶
To install the complete SearXNG suite into all LXC containers leave the container argument empty and run:
$ sudo -H ./utils/lxc.sh build
$ sudo -H ./utils/lxc.sh install suite
To build & install suite only in one container you can use by example:
$ sudo -H ./utils/lxc.sh build searxng-archlinux
$ sudo -H ./utils/lxc.sh install suite searxng-archlinux
The command above installs a SearXNG suite (see Installation Script). To install a nginx reverse proxy (or alternatively use apache):
$ sudo -H ./utils/lxc.sh cmd -- FORCE_TIMEOUT=0 ./utils/searxng.sh install nginx
Same operation just in one container of the suite:
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux FORCE_TIMEOUT=0 ./utils/searxng.sh install nginx
The FORCE_TIMEOUT environment is set to zero to run the script without user interaction.
To get the IP (URL) of the SearXNG service in the containers use show suite
command. To test instances from containers just open the URLs in your
WEB-Browser:
$ sudo ./utils/lxc.sh show suite | grep SEARXNG_URL
[searxng-ubu2110] SEARXNG_URL : http://n.n.n.170/searxng
[searxng-ubu2004] SEARXNG_URL : http://n.n.n.160/searxng
[searxnggfedora35] SEARXNG_URL : http://n.n.n.150/searxng
[searxng-archlinux] SEARXNG_URL : http://n.n.n.140/searxng
Clean up¶
If there comes the time you want to get rid off all the containers and clean up local images just type:
$ sudo -H ./utils/lxc.sh remove
$ sudo -H ./utils/lxc.sh remove images
Setup SearXNG buildhost¶
You can install the SearXNG buildhost environment into one or all containers. The installation procedure to set up a build host takes its time. Installation in all containers will take more time (time for another cup of coffee).
sudo -H ./utils/lxc.sh cmd -- ./utils/searxng.sh install buildhost
To build (live) documentation inside a archlinux container:
sudo -H ./utils/lxc.sh cmd searxng-archlinux make docs.clean docs.live
...
[I 200331 15:00:42 server:296] Serving on http://0.0.0.0:8080
To get IP of the container and the port number live docs is listening:
$ sudo ./utils/lxc.sh show suite | grep docs.live
...
[searxng-archlinux] INFO: (eth0) docs.live: http://n.n.n.140:8080/
Command Help¶
The --help
output of the script is largely self-explanatory:
usage::
lxc.sh build [containers|<name>]
lxc.sh copy [images]
lxc.sh remove [containers|<name>|images]
lxc.sh [start|stop] [containers|<name>]
lxc.sh show [images|suite|info|config [<name>]]
lxc.sh cmd [--|<name>] '...'
lxc.sh install [suite|base [<name>]]
build
:containers: build, launch all containers and 'install base' packages
:<name>: build, launch container <name> and 'install base' packages
copy:
:images: copy remote images of the suite into local storage
remove
:containers: delete all 'containers' or only <container-name>
:images: delete local images of the suite
start/stop
:containers: start/stop all 'containers' from the suite
:<name>: start/stop container <name> from suite
show
:info: show info of all (or <name>) containers from LXC suite
:config: show config of all (or <name>) containers from the LXC suite
:suite: show services of all (or <name>) containers from the LXC suite
:images: show information of local images
cmd
use single quotes to evaluate in container's bash, e.g.: 'echo $(hostname)'
-- run command '...' in all containers of the LXC suite
:<name>: run command '...' in container <name>
install
:base: prepare LXC; install basic packages
:suite: install LXC searxng suite into all (or <name>) containers
LXC suite: searxng
Suite includes installation of SearXNG
images: ubu2004 ubu2204 fedora35 archlinux
containers: searxng-ubu2004 searxng-ubu2204 searxng-fedora35 searxng-archlinux
SearXNG suite config¶
The SearXNG suite is defined in the file git://utils/lxc-searxng.env:
# -*- coding: utf-8; mode: sh indent-tabs-mode: nil -*-
# SPDX-License-Identifier: AGPL-3.0-or-later
# shellcheck shell=bash
# This file is a setup of a LXC suite. It is sourced from different context, do
# not manipulate the environment directly, implement functions and manipulate
# environment only in subshells.
lxc_set_suite_env() {
export LXC_SUITE_NAME="searxng"
# name of https://images.linuxcontainers.org
export LINUXCONTAINERS_ORG_NAME="${LINUXCONTAINERS_ORG_NAME:-images}"
export LXC_HOST_PREFIX="${LXC_SUITE_NAME:-searx}"
export LXC_SUITE=(
# end of standard support see https://wiki.ubuntu.com/Releases
"$LINUXCONTAINERS_ORG_NAME:ubuntu/20.04" "ubu2004" # LTS EOSS April 2025
"$LINUXCONTAINERS_ORG_NAME:ubuntu/22.04" "ubu2204" # LTS EOSS April 2027
# EOL see https://fedoraproject.org/wiki/Releases
"$LINUXCONTAINERS_ORG_NAME:fedora/35" "fedora35"
# rolling releases see https://www.archlinux.org/releng/releases/
"$LINUXCONTAINERS_ORG_NAME:archlinux" "archlinux"
)
}
lxc_suite_install_info() {
(
lxc_set_suite_env
cat <<EOF
LXC suite: ${LXC_SUITE_NAME}
Suite includes installation of SearXNG
images: ${LOCAL_IMAGES[*]}
containers: ${CONTAINERS[*]}
EOF
)
}
lxc_suite_install() {
(
lxc_set_suite_env
FORCE_TIMEOUT=0 "${LXC_REPO_ROOT}/utils/searxng.sh" install all
rst_title "Suite installation finished ($(hostname))" part
if ask_yn "Developer install? (wraps source from HOST into the running instance)" Yn; then
"${LXC_REPO_ROOT}/utils/searxng.sh" searxng.install.link_src "$(pwd)"
fi
lxc_suite_info
echo
)
}
lxc_suite_info() {
(
lxc_set_suite_env
for ip in $(global_IPs) ; do
if [[ $ip =~ .*:.* ]]; then
info_msg "(${ip%|*}) IPv6: http://[${ip#*|}]"
else
# IPv4:
# shellcheck disable=SC2034,SC2031
info_msg "(${ip%|*}) docs-live: http://${ip#*|}:8080/"
fi
done
"${LXC_REPO_ROOT}/utils/searxng.sh" searxng.instance.env
)
}