Complete Guide on How to Setup a Nextcloud Spreed Signaling Server (i.e. the Talk High-Performance Backend) on Debian (12) Bookworm

This is a tutorial on how to setup the High-performance backend for the Talk/Spreed app within a Nextcloud instance. While I found several tutorials online that guide through parts of the process, those I found were insufficient for allowing access to users behind strict firewalls or NATs, and additionally were mostly written for Ubuntu. Furthermore, since Debian Bookworm most services we require can be found in the Debian repositories for Bookworm, which makes it easier to setup the signaling server. This tutorial is an updated version from my tutorial on how to set up a nextcloud-spreed-signaling server on Debian 11, and aims at creating a fully functional High-Performance backend installed on Debian Bookworm, which also clients from within big organisations or companies can access.

This tutorial will explain how to prepare and set up a server to use the standalone spreed signaling server, plus, how to combine it with a TURN server. I have been hosting video conferences and workshops with for an academic audience, where the users often sit in universities or big organisation, and their internet traffic is limited by NAT and firewall settings. This is, because in big organisation sometimes only the most necessary ports for webbrowsing (53, 80, 443) are open, and the default ports of turn and signaling servers are outside that range. So in order to allow their access, both the TURN server and the signaling server need to be accessible on port 443. You can achieve that by either running the TURN server on a dedicated VPS, or by adding it to an existing server (such as the signaling server you are going to set up) while using an additional IPv4 address.

This tutorial will show both options, but most importantly, it will provide an inclusive setup to also allow access from clients behind strictly set firewalls.

Requirements:

• You will need a server (VPS) with Debian 12 minimal pre-installed for the signaling server.   
  (Nextcloud suggests to use a server with dedicated cores; For my purposes [<30 video conference participants], a small VPS with 2vCores and 2GB RAM has been sufficient so far)
• A Domain and access to the zone file (DNS settings)
• For the TURN server you will either need a second VPS or an additional IPv4 address that you can bind to an existing server (see chpt. 2 for more details), such as the signaling server you are going to set up.

Skill-Level:

• You know how to edit your DNS settings
• You know how to install a VPS
• You know how to open a Linux terminal

Text between the <less/more marks> indicate a variable you will create or choose (such as a user name or secret), but please leave out the marks themselves when entering the values into the configuration files.

1. Prepare Server

Let’s start with creating the DNS entries for your signaling and TURN servers. Go to your domain’s DNS settings and create two A entries, like

signaling A <XXX.XXX.XXX.XXX>

for your signaling server, and one for your TURN server like

turn A <YYY.YYY.YYY.YYY>

(Let’s do that at the beginning, so when we get to creating certificates, the subdomains should already be accessible). You may want to change the TTL to a low value, such as 600.

After renting your server, you should have gotten some root login credentials. Open the terminal on your local machine and SSH into your new server using those credentials:

ssh root@XXX.XXX.XXX.XXX

(Optionally) you may want to rename the hostname of your VPS:

hostnamectl set-hostname signaling.your.domain

First important step: Change your root password with

passwd

(If you want a random password, you can use pwgen 16; If you do, please write it down somewhere :-)

1.a) Create a SSH User

Create a new user

useradd -m <ssh-user>

Provide password

passwd <ssh-user>

Change the shell for the new user

chsh -s /bin/bash <ssh-user>

Add the user to the sudo group

usermod -a -G sudo <ssh-user>

And prepare for password-less ssh usage

su <ssh-user>

ssh-keygen

return to be root

exit

Lets make some changes to the ssh server to use a different than the default port, to disallow root login, limit ssh access to IPv4, and to allow password-less authentication at ssh login:

nano /etc/ssh/sshd_config

Uncomment (i.e. take out the "#" in front of a line), modify, or add the lines accordingly

Port 2222
PermitRootLogin no
AddressFamily inet
PubkeyAuthentication yes

and restart the ssh server

systemctl restart sshd

Now you can log out, or open a second ssh session on your desktop/local computer, pass on your ssh-key and, login again.

ON YOUR LOCAL MACHINE:

Create a ssh key

ssh-keygen (press "enter" when asked for a password to leave it empty)

and copy it to the VPS by using:

ssh-copy-id -i ~/.ssh/id_rsa.pub -p 2222 <ssh-user>@signaling.your.domain   

 

After that you can login without a password to your VPS

ssh -p 2222 <ssh-user>@signaling.your.domain

Go back into the ssh server configuration

sudo nano /etc/ssh/sshd_config

so you can disable password authentication for ssh, and only allow login for your <ssh-user>

PasswordAuthentication no
AllowUsers <ssh-user> 

Restart the ssh server

sudo systemctl restart sshd

While you could continue using sudo to make further changes, we will switch back to su for convenience

sudo -s

1.b) Firewalling and Brute Force Mitigation

Let's further enhance the VPS security by adding basic firewall and brute force mitigation

apt install ufw fail2ban -y

Configure the firewall to allow only incoming traffic via IPv4 to your ssh server

ufw allow proto tcp to 0.0.0.0/0 port 2222

and open port 443 for apache (and the TURN server)

ufw allow 443

ufw allow 80 #required for certbot

Enable the firewall

ufw enable

Increase brute force attack prevention, by creating a new file (for local settings)

nano /etc/fail2ban/jail.local

and add

[sshd] 
enable = true 
port = 2222 
logpath = %(sshd_log)s 
backend = %(sshd_backend)s
 
[sshd-ddos] 
enable = true 
port = 2222
logpath = %(sshd_log)s 
backend = %(sshd_backend)s

and restart fail2ban

systemctl restart fail2ban   
 

2. TURN server

According to the Nextcloud doumentation

A TURN server running on port 443 (or 80) is required in almost all scenarios

Even if the Nextcloud Talk High Performance Backend is used and publicly accessible, a TURN server might be needed:

The High Performance Backend uses a certain range of ports for WebRTC media connections (20000-40000 by default). A client could be behind a restrictive firewall that only allows connections to port 443, so even if the High Performance Backend is publicly accessible the client would need to connect to a TURN server in port 443, and the TURN server will then relay the packets to the 20000-40000 range in the High Performance Backend.

For maximum compatibility the TURN server should be configured to listen on port 443. Therefore, when both a TURN server and the High Performance Backend are used each one should run in its own server, or in the same server but each one with its own IP address, as the High Performance Backend will need to bind to port 443 too.

Hence, we can either run a dedicated VPS for the TURN server, or add a second IP to an existing and running VPS, such as the signaling server you are setting up. In the past, I used a dedicated VPS for this purpose, but my server load was too little to justify that (I had sessions with up to 30 participants at once, and the turnserver hardly used any memory or cpu).

IF YOU INSTALL YOUR TURN SERVER ON A DEDICATED VPS, REPEAT chpt. 1 ON THE SECOND VPS, AND JUMP TO 2.b)

2.a) Add additional IP address (Optional)

If you decided to use an additional IPv4, you will most likely have to add it to the network interfaces of the machine you want to add it, in the control panel of your VPS host.

To utilise an additional IPv4 address on an existing server, edit your existing config file or add a new one

nano /etc/network/interfaces.d/<your-config-file>.cfg

and add a new network interface

auto eth0:1
iface eth0:1 inet static
	address YYY.YYY.YYY.YYY
	netmask 255.255.255.255

If you use an existing VPS that already has apache installed, we can make the following modifications to apache; if not, then install apache now (which we will need later anyway) 

apt install apache

Make Apache to listen only to original IP address by editing

nano /etc/apache2/ports.conf

and add your original IP address (from the network interface eth0 in your /etc/network/interfaces.d/<your-config-file>.cfg or by looking it up using “hostname -I") in front of the ports apache is listening to

Listen XXX.XXX.XXX.XXX:80
<IfModule ssl_module>
	Listen XXX.XXX.XXX.XXX:443
</IfModule>
<IfModule mod_gnutls.c>
	Listen XXX.XXX.XXX.XXX:443
</IfModule>

Reboot server

reboot

Potentially you may have to shut down and restart the VPS from the control panel of your VPS host to enable the additional IPv4 address.

Login again

ssh -p 2222 <ssh-user>@signaling.your.domain

2.b) Install and configure coTURN

Install coTURN

apt install coturn -y

The coTURN service is executed as an unprivileged user like turnserver. Due to this by default coTURN can not use privileged ports, like port 443. Linux kernel capabilities could be used to overcome this limitation. Capabilities can be associated with executable files using setcap, so you could allow the /usr/bin/turnserver executable to bind sockets to privileged ports with:

setcap cap_net_bind_service=+ep /usr/bin/turnserver

Back up the original configuration file

cp /etc/turnserver.conf /etc/turnserver.conf.BAK

Create a "static-auth-secret" for you turnserver with

openssl rand -hex 32

and add it along with the other required configuration to

nano /etc/turnserver.conf

listening-port=443 
tls-listening-port=443
listening-ip=YYY.YYY.YYY.YYY
relay-ip=XXX.XXX.XXX.XXX
fingerprint 
use-auth-secret
static-auth-secret=<your-turnserver-secret-created-above>
realm=turn.your.domain
total-quota=0 
bps-capacity=0
stale-nonce
no-multicast-peers
no-stdout-log
log-file=/var/log/coturn.log
#verbose
simple-log
cert=/etc/coturn/certs/turn.your.domain/cert.pem 
pkey=/etc/coturn/certs/turn.your.domain/privkey.pem
no-tlsv1
no-tlsv1_1

If you need more log verbosity when testing your TURN server, uncomment the "#verbose" flag.

Restart your turn server

systemctl restart coturn

2.c) SSL Certificates with Letsencrypt

In order to use TLS encrypted data transfer, install certbot

apt-get install certbot

and create letsencrypt certificates

certbot certonly --standalone --preferred-challenges http --deploy-hook "systemctl restart coturn" -d turn.your.domain

Because coTURN doesn't have privileged rights, it is not allowed to read the certificates. Hence, create new folder where to copy the certificates to

sudo mkdir -p /etc/coturn/certs

Change the ownership and access rights for the coTURN user & group

sudo chown -R turnserver:turnserver /etc/coturn/

sudo chmod -R 700 /etc/coturn/

Because Letsecnrypt certificates are only valid for 3 months, they regularly need to be renewed. So you it is wise to automatise the certificate copy process. Create a script that can do that and place it in the renewal-hooks directory of certbot

nano /etc/letsencrypt/renewal-hooks/deploy/coturn-certbot-deploy.sh

#!/bin/sh
set -e
for domain in $RENEWED_DOMAINS; do
case $domain in
turn.your.domain)
daemon_cert_root=/etc/coturn/certs
 
# Make sure the certificate and private key files are
# never world readable, even just for an instant while
# we're copying them into daemon_cert_root.
umask 077
 
cp "$RENEWED_LINEAGE/fullchain.pem" "$daemon_cert_root/$domain.cert"
cp "$RENEWED_LINEAGE/privkey.pem" "$daemon_cert_root/$domain.key"
 
# Apply the proper file ownership and permissions for
# the daemon to read its certificate and key.
chown turnserver "$daemon_cert_root/$domain.cert" \
"$daemon_cert_root/$domain.key"
chmod 400 "$daemon_cert_root/$domain.cert" \
"$daemon_cert_root/$domain.key"
service coturn restart >/dev/null
;;
esac
done 

Make sure that  certbot will run once a week to check in time before your certs expire, and to get new ones. Run

crontab -e

And add

 22 2 * * 1 certbot certonly --standalone --preferred-challenges http --deploy-hook "systemctl restart coturn" -d turn.your.domain --quiet

While the simple log configuration of coTURN will overwrite the log file on every start-up, if your server is up and running for a while, we can configure the log to auto-rotate, by creating anew file

nano /etc/logrotate.d/coturn

and add

/var/log/coturn.log {
rotate 7
daily
missingok
notifempty
compress
postrotate
endscript
}

 3. Install Signaling Server

Since Debian 12 we can find the necessary components for the signaling server from the Debian repositories. However, the signaling server itself need to be install from the backports repository, as the version from stable is outdated. So first we have to add the backports repository

echo "deb http://deb.debian.org/debian/ bookworm-backports main" >> /etc/apt/sources.list

and then we can install the signaling server

apt update && apt install -t bookworm-backports nextcloud-spreed-signaling -y

as well as the Janus and NATS from the stable repos.

apt install janus nats-server -y

3.1 WebRTC Gateway: Janus

A Janus server (from https://github.com/meetecho/janus-gateway) can be used to act as a WebRTC gateway. To use it, we first have to create a turn-rest-api key

openssl rand -base64 16

and change the Janus configuration in

nano /etc/janus/janus.jcfg

Look for the section nat to add your turn-rest-api key and enable "full_trickle"

turn_rest_api_key = <turn-rest-api-key>
full_trickle = true

Edit

nano /etc/janus/janus.transport.http.jcfg

and in the section [general] set interface = "lo" 

Edit

nano /etc/janus/janus.transport.websockets.jcfg 

and in the section [general] set ws_interface = "lo"

Restart Janus service

systemctl restart janus

 The NATS-server does not require any further configuration.

3.2 Configure the Signaling Server

For the signaling server configuration we will need a couple of secrets, so first create those
 

Nextcloud Secret Key

openssl rand -hex 16

Block-Key

openssl rand -hex 16

Hash-Key

openssl rand -hex 16

Now we have five keys in total (incl. those from chapter 2 and 3)

Turn-Key from chpt. 2: <your-turnserver-secret-created-above>   
api-key from chpt. 3: <turn-rest-api-key>   
Nextcloud-Secret-Key: <nextcloud-secret-key>   
Block-Key: <block-key>   
Hash-Key: <hash-key>   
 

Let's back-up the original config file

mv /etc/nextcloud-spreed-signaling/server.conf /etc/nextcloud-spreed-signaling/server.conf.BAK

before creating the signaling server configuration file

nano /etc/nextcloud-spreed-signaling/server.conf

and copy, paste and adjust the file (change keys to yours)

[http]
listen = 127.0.0.1:8080

[app] 
debug = false

[sessions] 
hashkey = <hash-key>
blockkey = <block-key>

#You may define several nextcloud instances for which you want to use the signaling server, here we only add one
[backend]
backends = backend-1
allowall = false 
timeout = 10 
connectionsperhost = 8

#add your nextcloud instance
[backend-1] 
url = https://your.nextcloud.domain
secret = <nextcloud-secret-key>

[nats] 
url = nats://localhost:4222


[mcu] 
type = janus 
url = ws://localhost:8188

[turn] 
apikey = <turn-rest-api-key>
secret = <your-turnserver-secret-created-in-chapter-2> 
servers = turn:turn.your.domain:443?transport=tcp

Restart the signaling server

systemctl restart nextcloud-spreed-signaling

3.3 Reverse Proxy Server

As you can see from section [http] in the server config, the server listens to 127.0.0.1 on port 8080, so we (install and) configure a webserver and use it to proxy to the signaling server. You can use Nginx, Caddy, or some other server, personally I prefer Apache. If you haven't installed apache already (from chapter 2), you can do it now with

apt install apache2 -y

Create a virtual host by creating the file

nano /etc/apache2/sites-available/signaling.conf

and add

<VirtualHost _default_:443> 
  ServerAdmin admin@localhost 
  ServerName localhost 
  ServerAlias signaling.your.domain

  SSLEngine on

  # Use HSTS
  Header always set Strict-Transport-Security "max-age=63072000; preload"

  <FilesMatch "\.(cgi|shtml|phtml|php)$"> 
    SSLOptions +StdEnvVars 
  </FilesMatch> 
  <Directory /usr/lib/cgi-bin> 
    SSLOptions +StdEnvVars 
  </Directory>

  ProxyPass "/standalone-signaling/"  "ws://127.0.0.1:8080/"

  RewriteEngine On
  # Websocket connections from the clients. 
  RewriteRule ^/standalone-signaling/spreed$ - [L]
  # Backend connections from Nextcloud. 
  RewriteRule ^/standalone-signaling/api/(.*) http://127.0.0.1:8080/api/$1 [L,P] 
  SSLCertificateFile      /etc/ssl/certs/ssl-cert-snakeoil.pem 
  SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key

  ErrorLog ${APACHE_LOG_DIR}/error.log 
  CustomLog ${APACHE_LOG_DIR}/access.log combined 
</VirtualHost>

(NOTE: HSTS is not required for the signaling server, but you may add it to enhance transport security)

Enable the virtual host

a2ensite signaling.conf

as well as the required apache modules

a2enmod proxy ssl proxy proxy_http proxy_wstunnel rewrite headers

and restart apache

systemctl restart apache2

3.4 SSL/TLS Certificates

If you haven't already installed certbot from in chapter 2, then do it now; Certainly you will want to add the apache plugin for certbot:

apt install certbot python3-certbot-apache -y   
 

Run certbot

certbot

Select your new virtual host and let certbot install your Letsencrypt certificates.

Make sure that  certbot will run once a week to check in time before your certs expire, and to get new ones. Run

crontab -e

And add

 23 2 * * 1 certbot renew --quiet

at the bottom of the file.

4. Add Signaling and TURN Server to Nextcloud

Open your Nextcloud instance, login as admin, and go to the admin settings (https://your.nextcloud.domain/settings/admin/talk).

In section STUN servers, add

stun:  turn.your.domain:443

In section Turn servers, add

turn and turns turn.your.domain:443  <your-turnserver-secret-created-in-chapter-2>  UDP and TCP

In section High-performance backend   
https://signaling.your.domain/standalone-signaling  check "Validate SSL certificate" and in the field shared secret add <nextcloud-secret-key>

5. Problems? Check your Logs

If you don't see a green check behind the Turn and high-performance backend settings, check your settings.

• Make sure you used the created secrets in the right place
• Check your firewall settings
• check if all services are up and running
  • systemctl status apache2
  • systemctl status nextcloud-spreed-signaling
  • systemctl status janus
  • systemctl status nats-server
  • systemctl status coturn
     
• Check your logs
  • journalctl -u apache2 -f
  • journalctl -u nextcloud-spreed-signaling -f
  • journalctl -u janus -f
  • journalctl -u nats-server -f
  • journalctl -u coturn -f

!!! Happy Video Conferencing !!!

6. Sources:

This tutorial is based on my previous tutorial on how to set up the signaling server on Debian 11.

For my previous tutorial I followed a couple of good tutorials that covered at least parts of the process, and I want to thank the authors of those for sharing their knowledge! Yet, none of the tutorials I found covered my use case, so I compiled my own tutorial for easy reproduction. The resulting tutorial is quite compressed and doesn't provide all the details on the settings. If you want to know more about the background, you may want to check the sources I used for compiling this tutorial:

https://nextcloud-talk.readthedocs.io/en/latest/system-requirements/

https://nextcloud-talk.readthedocs.io/en/latest/TURN

https://github.com/strukturag/nextcloud-spreed-signaling

https://morph027.gitlab.io/blog/nextcloud-spreed-signaling/

https://nichteinschalten.de/signalisierungsserver-fuer-nextcloud-aufsetzen-how-to/

https://markus-blog.de/index.php/2020/07/30/how-to-install-nextcloud-talk-high-performance-backend-with-stun-turnserver-on-ubuntu/

https://serverfault.com/questions/849683/how-to-setup-coturn-with-letsencrypt

7. Updates:

18.01.2024
Building the signaling server now requires golang >1.20. Since the bookworm repository only contains version 1.19, golang needs to be collected from the book-backports repo. I have adjusted the tutorial accordingly.

08.02.2024
When the original version of this Debian12 tutorial was written, the signaling server version from the Debian repos was faulty and the tutorial described how to build the signaling server from the source. Since the release of the signaling server v.1.2.2.3 in the backports repository, we can use this version. The tutorial has thus been updated to reflect this change.

3.2 Build the Nextcloud Spreed Signaling Server from source

When I was first writing this tutorial, the signaling server package contained bugs so I explained how to build the signaling server from the source. I keep this here as reference.

Setup Nextcloud Spreed Signaling Server

At the time of writing, there is a nextcloud-spreed-signaling package in version 1.2.2 (which is actually the latest release) available as a package in the bookworm-backports repository (the version 0.4.1 in stable is outdated and won't work with recent Talk versions), but it appears to be buggy. One problem is that the systemd file for running the signaling server as daemon contains the wrong ExecPath (which can be fixed manually by overwriting the file /etc/systemd/system/multi-user.target.wants/nextcloud-spreed-signaling.service), the other problem I encountered was that the signaling server cannot connect with the janus-gateway.
 

However, I had no problems to get the signaling server to work when building the server from source, which is the approach used in this tutorial.

For the signaling server to be built, golang version >1.20 is required, which we can find from the backports repository. So we have to add it to the sources

echo "deb http://deb.debian.org/debian/ bookworm-backports main" >> /etc/apt/sources.list

Update and install golang

apt update && apt install -t bookworm-backports golang -y

Get other dependencies

apt install git automake build-essential python3 protobuf-compiler -y

Download the signaling server from github and build it

cd /opt 

git clone https://github.com/strukturag/nextcloud-spreed-signaling.git

cd nextcloud-spreed-signaling/ 

make build

 

Running the Signaling Server as Daemon

To run the signaling server as a daemon, copy the binary to Debian's default binary directory

cp bin/signaling /usr/bin/signaling

and create a new user that will run the signaling server

useradd --system --shell /usr/sbin/nologin --comment "Standalone signaling server for Nextcloud Talk." signaling

Create a configuration file for the signaling server with appropriate ownership settings

mkdir /etc/signaling/

touch /etc/signaling/server.conf

chown signaling: /etc/signaling/server.conf

chmod 600 /etc/signaling/server.conf

Create the systemd service file

nano /etc/systemd/system/signaling.service

and add

[Unit]
Description=Nextcloud Talk signaling server
After=janus.service

[Service]
ExecStart=/usr/bin/signaling --config /etc/signaling/server.conf
User=signaling
Group=signaling
Restart=on-failure

# Makes sure that /etc/signaling is owned by this service
ConfigurationDirectory=signaling

# Hardening - see systemd.exec(5)
CapabilityBoundingSet=
ExecPaths=/usr/bin/signaling /usr/lib
LockPersonality=yes
MemoryDenyWriteExecute=yes
NoExecPaths=/
NoNewPrivileges=yes
PrivateDevices=yes
PrivateTmp=yes
PrivateUsers=yes
ProcSubset=pid
ProtectClock=yes
ProtectControlGroups=yes
ProtectHome=yes
ProtectHostname=yes
ProtectKernelLogs=yes
ProtectKernelModules=yes
ProtectKernelTunables=yes
ProtectProc=invisible
ProtectSystem=strict
RemoveIPC=yes
RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX
RestrictNamespaces=no
RestrictRealtime=yes
RestrictSUIDSGID=yes
SystemCallArchitectures=native
SystemCallFilter=@system-service
SystemCallFilter=~ @privileged

[Install]
WantedBy=multi-user.target

 

Reload the daemon and enable the signaling server daemon

systemctl daemon-reload && systemctl enable signaling

 

18.10.2024
Corrected some file paths from [..]/signaling to [..]/nextcloud-spreed-signaling which was left in error after changing from the self-built signaling server to the debian package.