Some time ago, I replaced LastPass with the Bitwarden password manager for personal use.  I wanted something that had the features of LastPass, but could be self hosted.  Bitwarden checks all of those boxes with a really slick set of clients, a Docker based server package and a super responsive developer.

The Docker container comes with a really easy to use script to launch it, configure it…and update it.  I have a scheduled job that runs every night to update the Docker containers.  Unfortunately, I woke up this morning to an inaccessible Bitwarden web vault.  This one took a while to figure out.

Starting with a little background about my setup.  I have Bitwarden (and many other services) running behind a NetScaler Content Switch running on a NetScaler VPX appliance.  The main reason is to allow multiple services to share port 443 on my single public IP.  Although, it does also let me use all kinds of fun NetScaler security features on my public facing services.

PRTG monitoring showed that the Bitwarden web valut went down at the same time that the update script installed the recent 1.23.0 update. Trying to access the site with a browser just timed out with no response.  The site was available by directly hitting the backend web server.

Digging through the update notes, I found this commit.  Ok, that’s a pretty secure cipher suite.  The DEFAULT_BACKEND cipher suite that the NetScaler assigns to all services by default doesn’t include any of those ciphers.

Ok, easy enough to fix…  I create a new Cipher group and include all of the fancy new ciphers that Bitwarden wants to use.  Assign the new group to my load balancer service and….nothing.

After a lot of digging around in logs and traces, I think to take a look at the NetScaler supported ciphers list.  Read close, and you’ll find these notes:

1. The following curves are supported for ECDHE key exchange algorithms:
   • ECDHE 521 curve
   • ECDHE 384 curve
   • ECDHE 256 curve
   • ECDHE 224 curve For more information about the ECDHE ciphers supported on a NetScaler appliance, see Configuring ECDHE Ciphers.
2. AES-GCM/SHA2 ciphers are supported on both the front end and back end SSL entities on an MPX appliance. On an SDX appliance, an SSL chip must be assigned to the VPX instance for this support. AES-GCM/SHA2 ciphers are supported only on the front end SSL entities on a VPX appliance.
3. All ChaCha20-Poly1035 ciphers use a TLS pseudo random function (PSF) with the SHA-256 hash function.

If you combine numbers 2 and 3, that means that all of the ciphers in Bitwarden’s new Nginx configuration are not supported for backend services by a NetScaler VPX.

To fix, you need to edit the default.conf file in the Bitwarden Docker persistent location (/opt/bitwarden/bwdata/nginx/default.conf).  After backing up the file, look for the following line:

ssl_ciphers "ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256";

Replace that line with the original cipher set:

ssl_ciphers "ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4:@STRENGTH";

Then restart the bitwarden-nginx docker container.  Everything should be back up and running.

Since, Bitwarden is only accessible through the NetScaler, having a weaker cipher on the backend server should not weaken the overall security.  You should have a solid SSL configuration on your front end already.

---

EDIT

Bitwarden’s developer, Kyle, was kind enough to point out that the ciphers in the default.conf file will be reset each time the docker update script is run.  To deal with this, I have added a line to my cron script that runs the updates.  Below is the entire script I use with cron to update Bitwarden and reset the ciphers for NetScaler compatibility.

#!/bin/sh

# Update Docker container
cd /opt/bitwarden
./bitwarden.sh updateself > /root/updateself.log
./bitwarden.sh update > /root/update.log

# Replace new Nginx ciphers with old ones to maintiain NetScaler compatibility
sed -i '/ssl_ciphers/c\ ssl_ciphers "ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4:@STRENGTH";' /opt/bitwarden/bwdata/nginx/default.conf

./bitwarden.sh restart

exit 0