#1192 lead to the following error for me on Archlinux:
`TASK [matrix-base : Install host dependencies] *******************************************************************************************************************************
fatal: [matrix.***.de]: FAILED! => changed=false
msg: |-
failed to install systemd-timesyncd: error: target not found: systemd-timesyncd`
There is no package called `systemd-timesyncd` on Archlinux. The service is installed with the [`systemd`](https://archlinux.org/packages/core/x86_64/systemd/) package itself.
I suggest removing the `systemd-timesyncd` from 2453876eb9/roles/matrix-base/tasks/server_base/setup_archlinux.yml (L7)
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/1282
It's mostly due to Docker CE dropping its repositories (and support) for
Debian 9.
If one installs Docker manually (likely a package named `docker.io`), it
will likely still work.
In any case, Debian 9 is old and end-of-life now, so advertising support
for it is not productive.
* add timeout param for nginx proxy
default value matrix_nginx_proxy_request_timeout is 60s
* default matrix_nginx_proxy_request_timeout - 60s
* few more variables for request timeout
* Update nginx.conf.j2
* Update nginx.conf.j2
This bridge doesn't support SQLite anyway, so it's not necessary
to carry around configuration fields and code for migration from SQLite
to Postgres. There's nothing to migrate.
This configuration is supposed to be kept clean and not reference variables defined in other roles.
`group_vars/matrix_servers` redefines these to hook our various roles together.
The Github link is just a redirect to Tulir's own GitLab, so I replaced the self-build link
The docker container repository was rearranged hierarchically (dock.mau.dev/tulir/mautrix-facebook -> dock.mau.dev/mautrix/facebook)
Tagged versions have been made available, thus :latest -> :v0.3.1
Updated settings in template file:
* relay for any user
* user permissions only for HS domain users
Co-authored-by: Jan <31133207+Jaffex@users.noreply.github.com>
Permissions, when set in the template, will be augmented rahter than replaced when using matrix_mautrix_signal_configuration_extension_yaml. Therefore, permissions shall only be set in the defaults/vars.yml or in the HS specific vars.yml file
Per default relay bot functionality is disabled; the bridge user permissions depends on the relay bot, if enabled the base domain users are on level relay, else remain on user;
The attached code for the "Serving the Federation API with your certificates and matrix-nginx-proxy" section suggests using the matrix.<your-domain> certificate for the federation API as opposed to the necessary <your-domain> certificate for the federation to work. This can cause some confusion to readers.
Recently, documentation on Synapse has been changed from .rst to .md. Therefore, the current links for the purge history API were resulting in a 404 error.
Added an example for configuring the variables for the OpenID, that might fails because the variables are not defined for the playbook parsing, but Synapse's own config file parsing.
It's mildly annoying when trying to execute these scripts while logged
in as a regular user, as the missing execute permissions will hinder
autocompletion even when trying to use with sudo.
These shell scripts don't contain secrets, but may fail when ran by a
regular user. The failure is due to the lack of access to the /matrix
directory, and does not result in any damage.
This commit introduces a new role that downloads and installs the
prometheus community postgres exporter https://github.com/prometheus-community/postgres_exporter.
A new credential is added to matrix_postgres_additional_databases that
allows the exporter access to the database to gather statistics.
A new dashboard was added to the grafana role, with some refactoring
to enable the dashboard only if the new role is enabled.
I've included some basic instructions for how to enable the role in
the Docs section.
In terms of testing, I've tested enabling the role, and disabling
it to make sure it cleans up the container and systemd role.
Added a mid-sized VPS configuration with configuration changes to the PostgreSQL database config.
Deleted single quotes in one of the examples to unify the examples
I changed the conditional statement in prosody systemd template to bind the localhost port by default if people have set ```matrix_nginx_proxy_enabled == false ```.
Hopefully that should make it the default behaviour now.
Jitsi-meet enabled websockets by default, claiming better reliability.
Matrix-nginx-proxy configuration has been set up according to the
Prosody documentation: https://prosody.im/doc/websocket
**HSTS Preloading:**
In its strongest and recommended form, the [HSTS policy](https://www.chromium.org/hsts) includes all subdomains, and indicates a willingness to be “preloaded” into browsers:
`Strict-Transport-Security: max-age=31536000; includeSubDomains; preload`
**X-Xss-Protection:**
`1; mode=block` which tells the browser to block the response if it detects an attack rather than sanitising the script.
Expected to have regressed after https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/1008
This patch comes with its own downsides (as described in the comments
for matrix_prometheus_node_exporter_container_http_host_bind_port),
but at least there's:
- no security issue
- metrics remain readable from matrix-prometheus (even if the network metrics are inaccurate)
A better patch is certainly welcome.
Not sure why this had been done in the first place.
It doesn't make any sense.
There's no relation between matrix-nginx-proxy and
prometheus-node-exporter.
These old versions of TLS rely on MD5 and SHA-1, both now broken, and contain other flaws. TLS 1.0 is no longer PCI-DSS compliant and the TLS working group has adopted a document to deprecate TLS 1.0 and TLS 1.1.
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/963
This also simplifies Prerequisites, which is great.
It'd be nice if we were doing these checks in some optional manner
and reporting them as helpful messages (using
`matrix_playbook_runtime_results`), but that's more complicated.
I'd rather drop these checks completely.
This variable was previously undefined in the role and was only getting
defined via `group_vars/matrix_servers`.
We now properly initialize it (and its good default value) in the role
itself.
Fixes a regression caused by a5ee39266c.
If the user id and group id were different than 991:991
(which used to be a hardcoded default for us long ago),
there was a mismatch between what Synapse was trying to use (991:991)
and what it was actually started with (in `--user=..`). It was then
trying to change ownership, which was failing.
This was mostly affecting newer installations which were not using the
991:991 defaults we had long ago (since a1c5a197a9).
- add matrix_postgres_backup_databases to be build on top of matrix_postgres_additional_databases
- POSTGRES_DB is now directly set from matrix_postgres_backup_databases while building the templates/env-postgres-backup.j2
Self-checks against the .well-known URIs look for the HTTP header
"Access-Control-Allow-Origin" indicating that the remode endpoint
supports CORS. But the remote server is not required to include
said header in the response if the HTTP request does not include
the "Origin" header. This is in accordance with the specification
[1] stating: 'A CORS request is an HTTP request that includes an
"Origin" header.'
This is in fact true for Gitlab pages hosting and that's why the
issue was identified.
Let's specify "Origin" header in the respective uri tasks performing
the HTTP request and ensure a CORS request.
[1] https://fetch.spec.whatwg.org/#http-requests
We have a flow like this:
1. matrix.DOMAIN vhost (matrix-domain.conf)
2. matrix-synapse vhost (matrix-synapse.conf); or matrix-corporal container, if enabled
3. (optional) matrix-synapse vhost (matrix-synapse.conf), if matrix-corporal enabled
4. matrix-synapse container
We are setting `X-Forwarded-For` correctly in step #1, but were
overwriting it in step #2 with something inaccurate.
Not doing anything in step #2 is better than doing the wrong thing.
It's probably best if we append another reverse-proxy address there
though, although what we're doing now (with this patch) seems to yield
the correct result (when matrix-corporal is not enabled).
When matrix-corporal is enabled, we still seem to do the wrong thing for
some reason. It's something to be fixed later on.
During first setup postgres takes its time to get up and running, resulting in "postgres in startup" exceptions from synapse if you run without additional services that come in between. Hence suggesting increasing the time a bit to avoid having an error which heals itself and thus is hard to spot for newcomers.
People who were disabling matrix-nginx-proxy (in favor of their own
nginx webserver) and also overriding `matrix_federation_public_port`,
found that the generated nginx configuration still hardcoded `8448`,
which forced their nginx server to use that, regardless of the fact
that `matrix_federation_public_port` was pointing elsewhere.
We now allow for the in-container federation port to be configurable,
and also automatically wire things properly.
This reverts commit fd3d48bb6d.
Normally this environment variable gets referred to from `settings.json.docker`,
but we have our own full configuration, which hardcodes `"trustProxy": true`,
thus making this useless.
This has been pointed out here:
fd3d48bb6d (commitcomment-47403097)
don't spawn an extra container
run pg_dumpall within matrix-postgres instead, ensures correct version
store under /matrix so a backup of the folder will contain a DB dump
use absolute paths just in case something in the ENV is messed up
We're talking about a webserver running on the same machine, which
imports the configuration files generated by the `matrix-nginx-proxy`
in the `/matrix/nginx-proxy/conf.d` directory.
Users who run an nginx webserver on some other machine will need to do
something different.
This give us the possibility to run multiple instances of
workers that that don't expose a port.
Right now, we don't support that, but in the future we could
run multiple `federation_sender` or `pusher` workers, without
them fighting over naming (previously, they'd all be named
something like `matrix-synapse-worker-pusher-0`, because
they'd all define `port` as `0`).
The default default Ansible screen output encodes and prints error outputs as a hard to read dense line of JSON.
This patch changes the ansible-playbook command behavior for this project to output yaml instead.
This leads to much easier management and potential safety
features (validation). In the future, we could try to avoid port
conflicts as well, but it didn't seem worth the effort to do it now.
Our port ranges seem large enough.
This can also pave the way for a "presets" feature
(similar to `matrix_nginx_proxy_ssl_presets`) which makes it even easier
for people to configure worker counts.
The quotes around "host" for both `--pid` and `--net` were
causing trouble for me:
> docker: --pid: invalid PID mode.
and:
> docker: Error response from daemon: network "host" not found.
I've also changed the `-v` call to `--mount` for consistency with the
rest of the playbook.
Also includes the dashboards for Synapse and for Node Exporter.
Again has only been tested on debian amd64 so far, but the grafana docker image is available for arm64 and arm32. Nice.
Basic system stats, to show stuff the synapse metrics
can't show such as resource usage by bridges, etc
Seems to work fine as well.
This too has only been tested on debian amd64 so far
I felt that adding another variable was probably going to be the easiest way to do this. I may end up adding another variable to enable this feature, for consistency with some of the other things.
This passes any arguments given to 'matrix-postgres-cli' to the 'psql' command.
Examples:
$ # start an interactive shell connected to a given db
$ sudo matrix-postgres-cli -d synapse
$ # run a query, non-interactively
$ sudo matrix-postgres-cli -d synapse -c 'SELECT group_id FROM groups;'
If they do, our next playbook runs would simply revert it
and report "changed" for that task.
There's no benefit to letting the bridge spew a new config file.
This does not apply to the mautrix whatsapp bridge, because that one
is written in Go (not Python) and takes different flags. There's no
equivalent flag there.
Fixes a regression introduced in f6097fbba1, which was cauing Synapse
to die with this error message:
> ValueError: sender_localpart needs characters which are not URL encoded.
These are just defensive cleanup tasks that we run.
In the good case, there's nothing to kill or remove, so they trigger an
error like this:
> Error response from daemon: Cannot kill container: something: No such container: something
and:
> Error: No such container: something
People often ask us if this is a problem, so instead of always having to
answer with "no, this is to be expected", we'd rather eliminate it now
and make logs cleaner.
In the event that:
- a container is really stuck and needs cleanup using kill/rm
- and cleanup fails, and we fail to report it because of error
suppression (`2>/dev/null`)
.. we'd still get an error when launching ("container name already in use .."),
so it shouldn't be too hard to investigate.
Not specifying bind addresses for the worker resulted in this warning:
> synapse.app - 47 - WARNING - None - Failed to listen on 0.0.0.0, continuing because listening on [::]
Additionally, metrics listening only on 127.0.0.1 seems like a no-op.
Only having it accessible from within the container is likely not what
we intend. Changed that to all interfaces as well.
Whether it actually gets exposed or not depends on the systemd service
and `matrix_synapse_workers_container_host_bind_address`.
This switches the `docker exec` method of spawning
Synapse workers inside the `matrix-synapse` container with
dedicated containers for each worker.
We also have dedicated systemd services for each worker,
so this are now:
- more consistent with everything else (we don't use systemd
instantiated services anywhere)
- we don't need the "parse systemd instance name into worker name +
port" part
- we don't need to keep track of PIDs manually
- we don't need jq (less depenendencies)
- workers dying would be restarted by systemd correctly, like any other
service
- `docker ps` shows each worker separately and we can observe resource
usage
We do this by creating one more layer of indirection.
First we reach some generic vhost handling matrix.DOMAIN.
A bunch of override rules are added there (capturing traffic to send to
ma1sd, etc). nginx-status and similar generic things also live there.
We then proxy to the homeserver on some other vhost (only Synapse being
available right now, but repointing this to Dendrite or other will be
possible in the future).
Then that homeserver-specific vhost does its thing to proxy to the
homeserver. It may or may not use workers, etc.
Without matrix-corporal, the flow is now:
1. matrix.DOMAIN (matrix-nginx-proxy/matrix-domain.conf)
2. matrix-nginx-proxy/matrix-synapse.conf
3. matrix-synapse
With matrix-corporal enabled, it becomes:
1. matrix.DOMAIN (matrix-nginx-proxy/matrix-domain.conf)
2. matrix-corporal
3. matrix-nginx-proxy/matrix-synapse.conf
4. matrix-synapse
(matrix-corporal gets injected at step 2).
This removes some `multi-target.wants` symlinks as well, etc.
But despite systemd saying:
> Removed symlink /etc/systemd/system/matrix-synapse.service.wants/matrix-synapse-worker@appservice:0.service
.. I still see such symlinks tehre for me for some reason, so keeping the
code (below) to find & delete them still seems like a good idea.
There was a `matrix_nginx_proxy_enabled|default(False)` check, but:
- it didn't seem to work reliably for some reason (hmm)
- referring to a `matrix_nginx_proxy_*` variable from within the
`matrix-synapse` role is not ideal
- exposing always happened on `127.0.0.1`, which may not be good enough
for some rarer setups (where the own webserver is external to the host)
I guess it didn't hurt to do it until now, but it's not great serving
federation APIs on the client-server API port, etc.
matrix-corporal doesn't work yet (still something to be solved in the
future), but its firewalling operations will also be sabotaged
by Client-Server APIs being served on the federation port (it's a way to get around its firewalling).
If we load it at runtime, during matrix-synapse role execution,
it's good enough for matrix-synapse and all roles after that,
but.. it breaks when someone uses `--tags=setup-nginx-proxy` alone.
The downside of including this vars file like this in `setup.yml`
is that the variables contained in it cannot be overriden by the user
(in their inventory's `vars.yml`).
... but it's not like overriding these variables was possible anyway
when including them at runtime.
Some people run Coturn or Jitsi, etc., by themselves and disable it
in the playbook.
Because the playbook is trying to be nice and clean up after itself,
it was deleting these Docker images.
However, people wish to pull and use them separately and would rather
they don't get deleted.
We could make this configurable for the sake of this special case, but
it's simpler to just avoid deleting these images.
It's not like this "cleaning things up" thing works anyway.
As time goes on, the playbook gets updated with newer image tags
and we leave so many images behind. If one doesn't run
`docker system prune -a` manually once in a while, they'd get swamped
with images anyway. Whether we leave a few images behind due to the lack
of this cleanup now is pretty much irrelevant.
We log everything in systemd/journald for every service already,
so there's no need for double-logging, bridges rotating log files
manually and other such nonsense.
In short, this makes Synapse a 2nd class citizen,
preparing for a future where it's just one-of-many homeserver software
options.
We also no longer have a default Postgres superuser password,
which improves security.
The changelog explains more as to why this was done
and how to proceed from here.
I had intentionally held it back in 39ea3496a4
until:
- it received more testing (there were a few bugs during the
migration, but now it seems OK)
- this migration guide was written
While administering we will occasionally invoke this script interactively with the "non-interactive" switch still there, yet still sit at the desk waiting for 300 seconds for this timer to run out.
The systemd-timer already uses a 3h randomized delay for automatic renewals, which serves this purpose well.
The `mobile` branch got merged to `master`, which ends up becoming
`:latest`. It's a "rewrite" of the bridge's backend and only
supports a Postgres database.
We'd like to go back (well, forward) to `:latest`, but that will take
a little longer, because:
- we need to handle and document things for people still on SQLite
(especially those with external Postgres, who are likely on SQLite for
bridges)
- I'd rather test the new builds (and migration) a bit before
releasing it to others and possibly breaking their bridge
Brave ones who are already using the bridge with Postgres
can jump on `:latest` and report their experience.
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/756
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/737
I feel like timers are somewhat more complicated and dirty (compared to
cronjobs), but they come with these benefits:
- log output goes to journald
- on newer systemd distros, you can see when the timer fired, when it
will fire, etc.
- we don't need to rely on cron (reducing our dependencies to just
systemd + Docker)
Cronjobs work well, but it's one more dependency that needs to be
installed. We were even asking people to install it manually
(in `docs/prerequisites.md`), which could have gone unnoticed.
Once in a while someone says "my SSL certificates didn't renew"
and it's likely because they forgot to install a cron daemon.
Switching to systemd timers means that installation is simpler
and more unified.
This reverts commit 2a25b63bb6.
Looking at other roles, we trigger building regardless of this.
It's better to always trigger it, because it's less fragile.
If the build fails and we only trigger it on "git changes"
then we won't trigger it for a while. That's not good.
Triggering it each and every time may seem like a waste,
but it supposedly runs quickly due to Docker caching.
This variable has been useless since 2019-01-08.
We probably don't need to check for its usage anymore,
given how much time has passed since then, but ..
Before we potentially clone to that path, we'd better make sure it exists.
We also simplify `when` statements a bit.
Given that we're in `setup_install.yml`, we know that the bridge is enabled,
so there's no need to check for that.
Not sure if it breaks with them or not, but no other directive
uses quotes and the nginx docs show examples without quotes,
so we're being consistent with all of that.
The different configurations are now all lower case, for consistent
naming.
`matrix_nginx_proxy_ssl_config` is now called
`matrix_nginx_proxy_ssl_preset`. The different options for "modern",
"intermediate" and "old" are stored in the main.yml file, instead of
being hardcoded in the configuration files. This will improve the
maintainability of the code.
The "custom" preset was removed. Now if one of the variables is set, it
will use it instead of the preset. This will allow to mix and match more
easily, for example using all the intermediate options but only
supporting TLSv1.2. This will also provide better backward
compatibility.
While it's kind of nice having it, it's also somewhat raw
and unnecessary.
Having a good default and not even mentioning it seems better
for most users.
People who need a more exposed bridge (rare) can use
override the default configuration using
`matrix_mautrix_signal_configuration_extension_yaml`.
The answer to these is: it's good to have them in both places.
The role defines the obvious things it depends on (not knowing
what setup it will find itself into), and then
`group_vars/matrix_servers` "extends" it based on everything else it
knows (the homeserver being Synapse, whether or not the internal
Postgres server is being used, etc.)
We need to suppress systemd service-stopping requests in certain rare
cases like https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/771
That issue seems to describe a case, where a migration from mxisd to
ma1sd was happening (DB files had just been moved), and then we were
attemping to stop `matrix-ma1sd.service` so we could import that database into
Postgres. However, there's neither `matrix-mxisd.service`, nor
`matrix-ma1sd.service` after `migrate_mxisd.yml` had just run, so
stopping `matrix-ma1sd.service` was failing.
We've had people who get the impression that well-known = bad,
DNS SRV = good, and who try to use DNS SRV for server delegation.
While it's true that DNS SRV can be superior for high-availability
scenarios, it's much harder to set up and comes with its own potential
downsides.
Using the well-known method is more straightforward and is enough
for almost all of us. Throwing people into the deep for no good
reason is not nice. Hopefully wording is better after this patch.
Otherwise the postgres upgrade fails with the following error:
Unexpected templating type error occurred on ({{
[matrix_postgres_connection_username]
+
matrix_postgres_additional_databases|map(attribute='username')
}}
): can only concatenate list (not "generator") to list
Fixes a problem like this:
> File "/usr/lib/python3.8/site-packages/mautrix/bridge/e2ee.py", line 79, in __init__
> raise RuntimeError("Unsupported database scheme")
mautrix-python's e2ee.py module expects to find `postgres://` instead of
`postgresql://`.
Our old (base-path -> data-path) SQLite migration can't work otherwise.
It's probably not necessary to keep it anymore, but since we still do,
at least we should take care to ensure it works.
Raspbian doesn't seem to support arm64, so this is somewhat pointless
right now.
However, they might in the future. Doing this should also unify us
some more with `setup_debian.yml` with the ultimate goal of
eliminating `setup_raspbian.yml`.
Until now, we've only supported non-amd64 on Raspbian.
Seems like there are now people running Debian/Ubuntu on ARM,
so we were forcing them into amd64 Docker packages.
I've gotten a report that this change fixes support
for Ubuntu Server 20.04 on RPi 4B.
The breaking changes are properly documented in the CHANGELOG.md file.
The date used is specified as XXXX-XX-XX and should be modified at the
moment of merge.
A new variable called `matrix_nginx_proxy_ssl_config` is created for
configuring how the nginx proxy configures SSL. Also a new configuration
validation option and other auxiliary variables are created.
A new variable configuration called `matrix_nginx_proxy_ssl_config` is
created. This allow to set the SSL configuration easily using the
default options proposed by Mozilla. The default configuration is set to
"Intermediate", removing the weak ciphers used in the old
configurations.
The new variable can also be set to "Custom" for a more granular control.
This allows to set another three variables called:
- `matrix_nginx_proxy_ssl_protocols`,
- `matrix_nginx_proxy_ssl_prefer_server_ciphers`
- `matrix_nginx_proxy_ssl_ciphers`
Also a new task is added to validate the SSL configuration variable.
Revert "Correct inabillity for appservice-discord to connect"
This reverts commit 673e19f830.
While certain things do work even with such a local URL, sending
messages leads to an error like this:
> [DiscordBot] verbose: DiscordAPIError: Invalid Form Body
> avatar_url: Not a well formed URL.
Fixes https://github.com/Half-Shot/matrix-appservice-discord/issues/649
The sample configuration file for appservice-discord
c29cfc72f5/config/config.sample.yaml (L8)
explicitly says that we need a public URL.
Now that 0.7.2 is out, the Docker image supports Postgres
and we can do the (SQLite -> Postgres) migration.
I've also found out that we needed to fix up the `tokens.ex_date` column
data type a bit to prevent matrix-registration from raising exceptions
when comparing `datetime.now()` with `ex_date` coming from the database.
Example:
> File "/usr/local/lib/python3.8/site-packages/matrix_registration/tokens.py", line 58, in valid
> expired = self.ex_date < datetime.now()
> TypeError: can't compare offset-naive and offset-aware datetimes
In cases where pgloader is not enough and we need to do some additional
migration work after it, we can now use
`additional_psql_statements_list` and
`additional_psql_statements_db_name`.
This is to be used when migrating `matrix-registration`'s data at the
very least.
This switches us to a container image maintained by the
matrix-registration developer.
0.7.2 also supports a `base_url` configuration option we can use to
make it easier to reverse-proxy at a different base URL.
We still keep some workarounds, because of this issue:
https://github.com/ZerataX/matrix-registration/issues/47
We were running into conflicts, because having initialized
the roles (users) and databases, trying to import leads to
errors (role XXX already exists, etc.).
We were previously ignoring the Synapse database (`homeserver`)
when upgrading/importing, because that one gets created by default
whenever the container starts.
For our additional databases, it's a similar situation now.
It's not created by default as soon as Postgres starts with an empty
database, but rather we create it as part of running the playbook.
So we either need to skip those role/database creation statements
while upgrading/importing, or to avoid creating the additional database
and rely on the import for that. I've gone for the former, because
it's already similar to what we were doing and it's simpler
(it lets `setup_postgres.yml` be the same in all scenarios).
Auto-migration and everything seems to work. It's just that
matrix-registration cannot load the Python modules required
for talking to a Postgres database.
Tracked here: https://github.com/ZerataX/matrix-registration/issues/44
Until this gets fixed, we'll continue default to 'sqlite'.
I was thinking that it makes sense to be more specific,
and using `_postgres_` also separated these variables
from the `_database_` variables that ended up in bridge configuration.
However, @jdreichmann makes a good point
(https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/740#discussion_r542281102)
that we don't need to be so specific and can allow for other engines (like MySQL) to use these variables.
Regression since 2d99ade72f and 9bf8ce878e, respectively.
When SQLite is to be used, these bridges expect an `sqlite://`
connection string, and not a plain file name (path), like Appservice
Discord and mautrix-whatsapp do.
The only one that remains is `matrix_synapse_database_password`, but
that's something old and should be dealt with separately in the future
(unless it remains as it is).
Instead of passing the connection string, we can now pass a name of a
variable, which contains a connection string.
Both are supported for having extra flexibility.
Since we'll likely have generic SQLite database importing
via [pgloader](https://pgloader.io/) for migrating bridge
databases from SQLite to Postgres, we'd rather avoid
calling the "import Synapse SQLite database" command
as just `--tags=import-sqlite-db`.
Similarly, for the media store, we'd like to mention that it's
related to Synapse as well.
We'd like to be more explicit, so as to be less confusing,
especially in light of other homeserver implementations
coming in the future.
Using the result of `password_hash` works for creating them,
but authentication seems to be failing with some tools like pgloader.
It's possible that we're not escaping things properly somewhere.
Ideally, it'd be nice to solve that. But the easier (and still
relatively safe/good) solution is to just turn that password hash
into a UUID that's safe for passing around without worrying about
escaping.
People can toggle between them now. The playbook also defaults
to using SQLite if an external Postgres server is used.
Ideally, we'd be able to create databases/users in external Postgres
servers as well, but our initialization logic (and `docker run` command,
etc.) hardcode too many things right now.
While these modules are really nice and helpful, we can't use them
for at least 2 reasons:
- for us, Postgres runs in a container on a private Docker network
(`--network=matrix`) without usually being exposed to the host.
These modules execute on the host so they won't be able to reach it.
- these modules require `psycopg2`, so we need to install it before
using it. This might or might not be its own can of worms.
The tasks in `create_additional_databases.yml` will likely
ensure `matrix-postgres.service` is started, etc.
If no additional databases are defined, we'd rather not execute that
file and all these tasks that it may do in the future.
> Invalid data passed to 'loop', it requires a list, got this instead: matrix_postgres_additional_databases. Hint: If you passed a list/dict of just one element, try adding wantlist=True to your lookup invocation or use q/query instead of lookup.
Well, or working around it, as I've done in this commit (which seems
more sane than `wantlist=True` stuff).
To avoid needing to have `jq` installed on the machine, we could:
- try to run jq in a Docker container using some small image providing
that
- better yet, avoid `jq` altogether
Moving it above the "uninstalling" set of tasks is better.
Extracting it out to another file at the same time, for readability,
especially given that it will probably have to become more complex in
the future (potentially installing `jq`, etc.)
v0.7.0 is broken right now, because it calls
`/_matrix/client/r0/admin/register`, which is now at
`/_synapse/admin/v1/register`.
This has been fixed here: 6b26255fea
.. but it's not part of any release.
Switching to `master` (`docker.io/devture/zeratax-matrix-registration:latest`) until it gets resolved.
Reported upstream here: https://github.com/ZerataX/matrix-registration/issues/43
Starting with Docker 20.10, `--hostname` seems to have the side-effect
of making Docker's internal DNS server resolve said hostname to the IP
address of the container.
Because we were giving the mailer service a hostname of `matrix.DOMAIN`,
all requests destined for `matrix.DOMAIN` originating from other
services on the container network were resolving to `matrix-mailer`.
This is obviously wrong.
Initially reported here: https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/748
We normally try to not use the public hostname (and IP address) on the
container network and try to make services talk to one another locally,
but it sometimes could happen.
With this, we use a `matrix-mailer` hostname for the matrix-mailer
container. My testing shows that it doesn't cause any trouble with
email deliverability.
If a service is enabled, a database for it is created in postgres with a uniqque password. The service can then use this database for data storage instead of relying on sqlite.
The Docker 19.04 -> 20.10 upgrade contains the following change
in `/usr/lib/systemd/system/docker.service`:
```
-BindsTo=containerd.service
-After=network-online.target firewalld.service containerd.service
+After=network-online.target firewalld.service containerd.service multi-user.target
-Requires=docker.socket
+Requires=docker.socket containerd.service
Wants=network-online.target
```
The `multi-user.target` requirement in `After` seems to be in conflict
with our `WantedBy=multi-user.target` and `After=docker.service` /
`Requires=docker.service` definitions, causing the following error on
startup for all of our systemd services:
> Job matrix-synapse.service/start deleted to break ordering cycle starting with multi-user.target/start
A workaround which appears to work is to add `DefaultDependencies=no`
to all of our services.
After recently updating my matrix-docker-ansible-deploy installation, matrix-appservice-discord would refuse to start, logging ECONNREFUSED to https://matrix.[mydomain]:443, which was resolving to 172.18.0.2 due to the `--hostname` in mailer grabbing that hostname.
Curious why the IRC bridge didn't have this issue, I looked into it, and it was connecting to `http://matrix-synapse:8008`. Correcting this one to that URL resolved the issue.
ma1sd requires the openid endpoints for certain functionality.
Example: 90b2b5301c/src/main/java/io/kamax/mxisd/auth/AccountManager.java (L67-L99)
If federation is disabled, we still need to expose these openid APIs on the
federation port.
Previously, we were doing similar magic for Dimension.
As per its documentation, when running unfederated, one is to enable
the openid listener as well. As per their recommendation, people
are advised to do enable it on the Client-Server API port
and use the `federationUrl` variable to override where the federation
port is (making federation requests go to the Client-Server API).
Because ma1sd always uses the federation port (unless you do some
DNS overwriting magic using its configuration -- which we'd rather not
do), it's better if we just default to putting the `openid` listener
where it belongs - on the federation port.
With this commit, we retain the "automatically enable openid APIs" thing
we've been doing for Dimension, but move it to the federation port instead.
We also now do the same thing when ma1sd is enabled.
We've had a report of the `connection` value getting cut off,
supposedly because it contains something that breaks off the string.
Using `|to_json` takes care of it.
This may be a bit premature, because the bridge didn't work for me
the last time I tried it (RC3).
Some bugs have been fixed to make our config compatible with v1.0.0
though, so it may work for some people (especially those starting
fresh).
I'm not for shipping potentially broken things, but given that we were
using `docker.io/halfshot/matrix-appservice-discord:latest` and that
points to v1.0.0 already (with no other tag we can use), our setup was
already broken in any case.
Now, at least it has some chance of running.
Many people probably didn't even know this - that ansible can be
quite a bit picky about what it will be willing to work with remotely.
Thanks @maxklenk !
Some people requested that `--tags=start` not set up service autostart.
One can now do `--tags=start --extra-vars="matrix_services_autostart_enabled=false"`
to just start services ones and not set up autostarting.
It's not like it worked anyway, because we don't have the necessary
services installed for transcription (Jigasi), nor recording (Jibri).
Disabling these, should hopefully disable their related elements
in the Jitsi Web UI.
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/726
This supersedes/fixes-up this Pull Request:
https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/719
The Jitsi Web and JVB containers now (in build 5142) always
start by bulding their own default configuration
(`config.js` and `sip-communicator.properties`, respectively).
The fact that we were generating these files ourselves was no longer of use,
because our configuration was thrown away in favor of the one created
by the containers on startup.
With this commit, we're completely redoing things. We no longer
generate these configuration files. We try to pass the proper
environment variables, so that Jitsi services can generate the
configuration files themselves.
Besides that, we try to use the "custom configuration" mechanism
provided by Jitsi Web and Jitsi JVB (`custom-config.js` and
`custom-sip-communicator.properties`, respectively), so that
we and our users can inject additional configuration.
Some configuration options we had are gone now. Others are no longer
controllable via variables and need to be injected using
the `_config_extension` variables that we provide.
The validation logic that is part of the role should take care
to inform people about how to upgrade (if they're using some custom
configuration, which needs special care now). Most users should not
have to do anything special though.
Since the switch from `-v` to `--mount` (in 1fca917ad1),
we've regressed when `matrix_ssl_retrieval_method == 'none'`.
In such a case, we don't create `/matrix/ssl` directories at all
and shouldn't be trying to mount them into the `matrix-nginx-proxy`
container.
Previously, with `-v`, Docker would auto-create them, effectively hiding
our mistake. Now that `--mount` doesn't do such auto-creation magic,
the `matrix-nginx-proxy` container was failing to start.
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/734
`-v` magically creates the source destination as a directory,
if it doesn't exist already. We'd like to avoid this magic
and the potential breakage that it might cause.
We'd rather fail while Docker tries to find things to `--mount`
than have it automatically create directories and fail anyway,
while having contaminated the filesystem.
There's a lot more `-v` instances remaining to be fixed later on.
This is just some start.
Things like `matrix_synapse_container_additional_volumes` and
`matrix_nginx_proxy_container_additional_volumes` were not changed to
use `--mount`, as options for each one are passed differently
(`ro` is `ro`, but `rw` doesn't exist and `slave` is `bind-propagation=slave`).
To avoid breaking people's custom volume mounts, we keep it as it is for now.
A deficiency with `--mount` is that it lacks the `z` option (SELinux
ownership changes), and some of our `-v` instances use that. I'm not
sure how supported SELinux is for us right now, but it might be,
and breaking that would not be a good idea.
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/716
This patch makes us use more fully-qualified container image names
(either prefixed with docker.io/ or with localhost/).
The latter happens when self-building is enabled.
We've recently had issues where if an image was removed manually
and the service was restarted (making `docker run` fetch it from Docker Hub, etc.),
we'd end up with a pulled image, even though we're aiming for a self-built one.
Re-running the playbook would then not do a rebuild, because:
- the image with that name already exists (even though it's something
else)
- we sometimes had conditional logic where we'd build only if the git
repo changed
By explicitly changing the name of the images (prefixing with localhost/),
we avoid such confusion and the possibility that we'd automatically pul something
which is not what we expect.
Also, I've removed that condition where building would happen on git
changes only. We now always build (unless an image with that name
already exists). We just force-build when the git repo changes.
We'd like the roles to be self-contained (as much as possible).
Thus, the `matrix-nginx-proxy` shouldn't reference any variables from
other roles. Instead, we rely on injection via
`group_vars/matrix_servers`.
Related to #681 (Github Pull Request)
Having it unset in the role itself (while referencign it) is a little strange.
Now people can look at the `roles/matrix-dynamic-dns/defaults/main.yml`
file and figure out everything that's necessary to run the role.
Related to #681 (Github Pull Request)
This broke in 63a49bb2dc.
Proxying the OpenID Connect endpoints is now possible,
but needs to be enabled explicitly now.
Supersedes #702 (Github Pull Request).
This patch builds up on the idea from that Pull Request,
but does things in a cleaner way.
We do this to match Synapse's new default "max_upload_size" (50MB).
This `matrix_nginx_proxy_proxy_matrix_client_api_client_max_body_size_mb`
default value only affects standalone usage of the `matrix-nginx-proxy`
role. When the role is used in the context of the playbook,
the value is dynamically assigned from `group_vars/matrix_servers`.
Somewhat related to #692 (Github Issue).
The regex introduced in 63a49bb2dc seems to take precedence
over the bare location blocks, causing a regression.
> It is important to understand that, by default, Nginx will serve regular expression matches in preference to prefix matches.
> However, it evaluates prefix locations first, allowing for the administer to override this tendency by specifying locations using the = and ^~ modifiers.
Source: https://www.digitalocean.com/community/tutorials/understanding-nginx-server-and-location-block-selection-algorithms
While v1.22.0 supposedly has multi-arch Docker images
(thanks to https://github.com/matrix-org/synapse/pull/7921),
I can't them on Docker Hub yet, so I'm backing out of this change
for now and letting people fall back to self-building there.
also, worker.yaml.j2:
- hone worker_name
- remove worker_pid_file entry (would only be used if worker_daemonize
set to true; also, synapse only knows about the container namespace
and thus can not provide the required host-view PID)
- cherry-pick "Ensure worker config exists in systemd service (#7528)"
from synapse d74cdc1a42e8b487d74c214b1d0ca575429d546a:
"check that the worker config file exists instead of silently failing."
If the SQLite database was from an older version of Synapse, it appears
that Synapse would try to run migrations on it first, before importing.
This was failing, because the file wasn't writable.
Hopefully, this fixes the problem.
Interestingly, no one has reported this failure before #662 (Github
Issue).
It doesn't make sense to keep saying that we support such old Ansible
versions, when we're not even testing on anything close to those.
Time is also passing and such versions are getting more and more
ancient. It's time we bumped our requirements to something that is more
likely to work.
showLabsSettings is the new enableLabs I guess. enableLabs doesn't seem to do anything anymore. It had been deprecated for a while.
This PR also removes @riot-bot:matrix.org as the default welcome_user_id since it doesn't exist anymore.
Change anonymized to more proper term as server vice this is not anonymized. Server name is the first parameter that is collected. And if server happens to be for individual use these statistics would be at personal level without any anonymising.
We recently had a report of the Postgres backup container's log file
growing the size of /var/lib/docker until it ran out of disk space.
Trying to prevent similar problems in the future.
Certain more-minimal Debian installations may not have
lsb-release installed, which makes the playbook fail.
We need lsb-release on Debian, so that ansible_lsb
could tell us if this is Debian or Raspbian.
In #628 I proposed a CORS change that turns out not to be the root of the issue. Caffeine-addled diagnosis leads to sloppy thinking, and this change should be reverted. In fact, if left it will cause problems for new installations.
Even with the v2 updates listed in #503 and partially addressed in #614, this is still needed to enable identity services to function with Element Desktop/Web. Testing on multiple clients with a clean config has confirmed this, at least for my installation.
Stating "many times" makes me think there is a finite amount of times I can run it before it *will* start causing trouble. But this is not true. So just state it can me ran more than once without causing trouble :)
Fix regression since 2a50b8b6bb (#597).
Dimension is intended to be embedded in various clients,
be it the Element service that we host (at element.DOMAIN),
some other Element (element-desktop running locally), etc.
The when statement is supposed to be on the block, not on the individual task.
It affects all tasks within the block (they're all to be executed when ma1sd is enabled and self-building is requested0.
The tag format used in the `ma1sd` repo have change. Versions no longer
start with 'v', and when building for non-amd64, we also need to strip
off the '-$arch' bit from the Docker image name.
Further, when building the .jar file, `ma1sd` currently names the .jar
based on the project's directory, which we call 'docker-src'. This means
other parts of the `ma1sd` build can't find the .jar file. Remedy this
by ensuring that the dir is called `docker-src/ma1sd`.
If a Postgres dump contains ALTER TABLE ... OWNER_TO <username>
statements which set the owner to a username different from
'synapse' the post Postgres import task will fail complaining
about lack of role.
Changing the matrix_postgres_connection_username group var has no
effect. However, the ALTER TABLE statements (and accompanying comments)
can be rewritten to change the username to 'synapse', which permits the
import task to succeed.
From a sample of 1, having the owner set in this was causes no
discernable side effects on the homeserver.
One could also remove the two variables from the docs completely,
because they are set by the playbook automatically.
Error: javax.net.ssl.SSLPeerUnverifiedException: Certificate for
<matrix.<your-domain>> doesn't match any of the subject alternative
names: [<your-domain>]
Fixes#577 (Github Issue).
`matrix_container_images_self_build` was not really doing anything
anymore. It previously was influencing `matrix_*_self_build` variables,
but it's no longer the case since some time ago.
Individual `matrix_*_self_build` variables are still available.
People that would like to toggle self-building for a specific component
ought to use those.
These variables are also controlled automatically (via
`group_vars/matrix_servers`) depending on `matrix_architecture`.
In other words, self-building is being done automatically for
all components when they don't have a prebuilt image for the specified
architecture. Some components only support `amd64`, while others also
have images for other architectures.
There's no change in the source code. Just a release bump for packaing
reasons. It doesn't matter much for us here, but let's be on the latest
tag anyway.
Postgres setup like
matrix_mautrix_telegram_configuration_extension_yaml: |
appservice:
database: "postgres://XXX:XXX@matrix-postgres:5432/mxtg"
will fail without the right Dockernetwork
`/etc/nginx/conf.d/default.conf` was previously causing
some issues when used with our `--user`.
It's not the case anymore, so we can remove it.
Fixes#369 (Github Issue).
Depending on the distro, common commands like sleep and chown may either
be located in /bin or /usr/bin.
Systemd added path lookup to ExecStart in v239, allowing only the
command name to be put in unit files and not the full path as
historically required. At least Ubuntu 18.04 LTS is however still on
v237 so we should maintain portability for a while longer.
the current version fails the import, because the volume for the media is missing. It still fails if you have the optional shared secret password provider is enabled, so that might need another mount. Commenting out the password provider in the hoimeserver.yaml during the run works as well.
This flag is necessary for Ansible to ask for the sudo password when using the non-root option as documented in the `hosts` file. Otherwise, Ansible errors out with `missing sudo password`.
This is mostly here to guard against problems happening
due to server migration and doing `chown -R matrix:matrix /matrix`.
Normally, the file is owned by `1000:1000`, as expected.
If ownership changes, Dimension could still start, but it will fail the
first time it tries to write to the database. Explicitly chowning
before startup guards against this.
Related to #485 and #486 (Github Pull Requests).
Also related to ccc7aaf0ce.
Dimension runs as the `node` user in the container (`1000:1000`).
It doesn't seem like we have a way around it. Thus, its configuration
must also be readable by that user (or group, in this case).
We don't really need to fail in such a spectactular way,
but it's probably good to do. It will only happen for people
who are defining their own user/group id, which is rare.
It seems like a good idea to tell them that this doesn't work
as they expect anymore and to ask them to remove these variables,
which otherwise give them a fake sense of hope.
Related to #486 (Github Pull Request).
If one runs the playbook with `--tags=setup-all`, it would have been
fine.
But running with a specific tag (e.g. `--tags=setup-riot-web`) would
have made that initialization be skipped, and the `matrix-riot-web` role
would fail, due to missing variables.
Ansible will migrate the ownership of the base path and config path, but
manual intervention will be required in order to migrate the ownership
of files in those directories (i.e. dimension.db).
Stop the services:
(local)$ ansible-playbook -i inventory/hosts setup.yml --tags=stop
Fix the permissions on the server:
(server)# chown -Rv "{{ matrix_user_username }}:{{ matrix_user_username }}" "{{ matrix_dimension_base_path }}"
which would typically look like:
(server)# chown -Rv matrix:matrix /matrix/dimension/
Reconfigure Dimension and start the services:
(local)$ ansible-playbook -i inventory/hosts setup.yml --tags=setup-dimension,start
* add permalinkPrefix to riot-web config
* add feature to change default theme of riot-web via its config file
* remove matrix_riot_web_change_default_theme and provide sane default
· 😅 How to keep this in sync with the matrix-synapse documentation?
· regex location matching is expensive
· nginx syntax limit: one location only per block / statement
· thus, lots of duplicate statements in this file
Well, actually 8cd9cde won't work, unless we put the
`|to_nice_yaml` thing on a new line.
We can, but that takes more lines and makes things look uglier.
Using `|to_json` seems good enough.
The whole file is parsed as YAML later on and merged with the
`_extension` variable before being dumped as YAML again in the end.
Hopefully fixes an error like this (which I haven't been able to
reproduce, but..):
> [modules/xmpp/strophe.util.js] <Object.i.Strophe.log>: Strophe: Error: Failed to construct 'RTCPeerConnection': 'matrix.DOMAIN' is not one of the supported URL schemes 'stun', 'turn' or 'turns'.
We define this password in the `sip-communicator.properties`
configuration file, so this is not needed for actually running JVB.
However, it does a (useless) safety check during container startup,
and we need to make that check happy.
We do this for 2 reasons:
- so we can control things which are not controllable using environment
variables (for example `stunServers` in jitsi/web, since we don't wish
to use the hardcoded Google STUN servers if our own Coturn is enabled)
- so playbook variable changes will properly rebuild the configuration.
When using Jitsi environment variables, the configuration is only built
once (the first time) and never rebuilt again. This is not the
consistent with the rest of the playbook and with how Ansible operates.
We're not perfect at it (yet), because we still let the Jitsi containers
generate some files on their own, but we are closer and it should be
good enough for most things.
Related to #415 (Github Pull Request).
I am new to synapse. Thanks so much for these playbooks!
I wasn't sure how you actual activate dimensions after setting these variables. Should you re run
ansible-playbook -i inventory/hosts setup.yml --tags=setup-all
ansible-playbook -i inventory/hosts setup.yml --tags=start
? If so perhaps you could tell the readers to do this somewhere in the help file
Thanks =]
This keeps the roles cleaner and more independent of matrix-base,
which may be important for people building their own playbook
out of the individual roles and not using the matrix-base role.
This adds into the Riot config.json the field
'default_server_config.m.homeserver.server_name'
with, by default, the value of the playbook's 'matrix_domain' variable.
Riot displays this string in its login page and will now say 'Sign in to
your Matrix account on example.org' (the server name) instead of 'Sign
in ... on matrix.example.org' (the server domain-name).
This string can be configured by setting the playbook variable
'matrix_riot_web_default_server_name'
to any string, so we can make Riot say for example 'Sign in ... on Our
Server'.
This fixes an incorrect indentation in the database specification for
appservice-irc which caused matrix-appservice-irc to refuse to start
with the remarkably unhelpful error message:
```
ERROR:CLI Failed to run bridge.
```
This also updates doc links to the new matrixdotorg repo because the
tedomum repo contains out-of-date documentation.
Synapse v1.9.0 changed some things which made the REST Auth Password
Provider break.
The ma1uta/matrix-synapse-rest-password-provider implements some
workarounds for now and will likely deliver a proper fix in the future.
Not much has changed between the 2 projects, so this should be a
painless transition.
This change allows us to work with both our existing Docker image
(`tedomum/matrix-appservice-irc:latest`) and with the
official Docker image (`matrixdotorg/matrix-appservice-irc`).
The actual change to the official Docker image requires more testing
and will be done separately.
Can you double check that the way I have this set only exposes it locally? It is important that the manhole is not available to the outside world since it is quite powerful and the password is hard coded.
Switching to the official image (vectorim/riot-web) should ensure:
- there's less breakage, as it's maintained by the same team as riot-web
- there's fewer actors we need to trust
- we can upgrade riot-web faster, as newer versions should be released
on Docker hub at the same time riot-web releases are made
Riot used to be fine with it being blank but now it complains. This creates an ugly looking comma when there is an identity server configured but I guess that's fine.
Prompted by: https://matrix.org/blog/2019/11/09/avoiding-unwelcome-visitors-on-private-matrix-servers
This is a bit controversial, because.. the Synapse default remains open,
while the general advice (as per the blog post) is to make it more private.
I'm not sure exactly what kind of server people set up and whether they
want to make the room directory public. Our general goal is to favor
privacy and security when running personal (family & friends) and corporate
homeservers, both of which likely benefit from having a more secure default.
Don't mention systemd-journald adjustment anymore, because
we've changed log levels to WARNING and Synapse is not chatty by default
anymore.
The "excessive log messages may get dropped on CentOS" issue no longer
applies to most users and we shouldn't bother them with it.
matrix_synapse_storage_path is already defined in matrix-synapse/defaults/main.yml (with a default of "{{ matrix_synapse_base_path }}/storage"), but was not being used for its presumed purpose in matrix-synapse.service.j2. As a result, if matrix_synapse_storage_path was overridden (in a vars.yml), the synapse service failed to start.
Invitations weren't working for me until I added 'nocanon' to these additional places. Until then, invitations failed with "Invalid signature for server ..." errors, as in https://github.com/matrix-org/synapse/issues/3294 .
I didn't check whether the user_directory/search proxy line also needs it, I just assumed it should have it too.
The other two proxy lines in this example also include a 'retry=0' parameter. That's a separate issue; I haven't touched it here.
Discord client IDs are numeric (e.g. 12345).
Passing them as integers however, causes the Discord bridge's YAML parser
to parse them as integers and its config schema validation will fail.
Fixes#240 (Github Issue)
This only gets triggered if:
- the Synapse role is used standalone and the default values are used
- the whole playbook is used, with `matrix_mxisd_enabled: false`
Continuation of #234 (Github Pull Request).
I had unintentionally updated the documentation for the feature,
saying the page is available at `https://matrix.DOMAIN/nginx_status`.
Looks like it wasn't the case, going against my expectations.
I'm correcting this with this patch.
The status page is being made available on both HTTP and HTTPS.
Serving over HTTP is likely necessary for services like
Longview
(https://www.linode.com/docs/platform/longview/longview-app-for-nginx/)
# Auth server config
auth:
# Publicly accessible base URL for the login endpoints.
# The prefix below is not implicitly added. This URL and all subpaths should be proxied
# or otherwise pointed to the appservice's webserver to the path specified below (prefix).
# This path should usually include a trailing slash.
public: http://example.com/login/
# Internal prefix in the appservice web server for the login endpoints.
prefix: /login
It's been pointed out that DEBUG logs could contain sensitive
information (access tokens, etc.), which makes them unsuitable
for sharing with others. INFO should be enough.
Also discussed previously in #213 (Github Pull Request).
shared-secret-auth and rest-auth logging is still at `INFO`
intentionally, as user login events seem more important to keep.
Those modules typically don't spam as much.
We recently had someone in the support room who set it to `false`
and the playbook ran without any issues.
This currently seems to yield the same result as 'none', but it's
better to avoid such behavior.
It adds support for a new `DISABLE_SENDER_VERIFICATION` environment
variable that can be used to disable verification of sender addresses.
It doesn't matter for us, but we upgrade to keep up with latest.
Looks like these client ids are actually integers,
but unless we pass them as a string, the bridge would complain with
an error like:
{"field":"data.auth.clientID","message":"is the wrong type","value":123456789012345678,"type":"string","schemaPath":["properties","auth","properties","clientID"]}
Explicitly-casting to a string should fix the problem.
The Discord bridge should probably be improved to handle both ints and
strings though.
We do restart Synapse explicitly, but some other services
(bridges, matrix-corporal, ..) may not restart sometimes.
It's best to restart all services explicitly.
Regression since 174a6fcd1b, #204 (Github Pull Request),
which only affects new servers.
Old servers which had their passkey.pem file relocated were okay.
ef5e4ad061 intentionally makes us conform to
the logging format suggested by the official Docker image.
Reverting this part, because it's uglier.
This likely should be fixed upstream as well though.
Somewhat related to #213 (Github Pull Request).
We've been moving in the opposite direction for quite a long time.
All services should just leave logging to systemd's journald.
Fixes a regression introduced during the upgrade to
Synapse v1.1.0 (in 2b3865ceea).
Since Synapse v1.1.0 upgraded to Python 3.7
(https://github.com/matrix-org/synapse/pull/5546),
we need to use a different modules directory when mounting
password provider modules.
Well, `config.yaml` has been playbook-managed for a long time.
It's now extended to match the default sample config of the Discord
bridge.
With this patch, we also make `registration.yaml` playbook-managed,
which leads us to consistency with all other bridges.
Along with that, we introduce `./config` and `./data` separation,
like we do for the other bridges.
According to
https://passlib.readthedocs.io/en/stable/lib/passlib.hash.sha512_crypt.html:
> salt (str) – Optional salt string. If not specified, one will be autogenerated (this is recommended).
> If specified, it must be 0-16 characters, drawn from the regexp range [./0-9A-Za-z].
Until now, we were using invalid characters (like `-`). We were also
going over the requested length limit of 16 characters.
This is most likely what was causing `ValueError` exceptions for some people,
as reported in #209 (Github Issue).
Ansible's source code (`lib/ansible/utils/encrypt.py`) shows that Ansible tries
to use passlib if available and falls back to Python's `crypt` module if not.
For Mac, `crypt.crypt` doesn't seem to work, so Ansible always requires passlib.
Looks like crypt is forgiving when length or character requirements are
not obeyed. It would auto-trim a salt string to make it work, which means
that we could end up with the same hash if we call it with salts which aer only
different after their 16th character.
For these reasons (crypt autotriming and passlib downright complaining),
we're now using shorter and more diverse salts.
I've been thinking of doing before, but haven't.
Now that the Whatsapp bridge does it (since 4797469383),
it makes sense to do it for all other bridges as well.
(Except for the IRC bridge - that one manages most of registration.yaml by itself)
appservice-irc doesn't have permission to create files in its project
directory and the intention is to log to the console, anyway. By
commenting out the file names, appservice-irc won't attempt to open the
files.
This means we need to explicitly specify a `media_url` now,
because without it, `url` would be used for building public URLs to
files/images. That doesn't work when `url` is not a public URL.
Until now, if `--tags=setup-synapse` was used, bridge tasks would not
run and bridges would fail to register with the `matrix-synapse` role.
This means that Synapse's configuration would be generated with an empty
list of appservices (`app_service_config_files: []`).
.. and then bridges would fail, because Synapse would not be aware of
there being any bridges.
From now on, bridges always run their init tasks and always register
with Synapse.
For the Telegram bridge, the same applies to registering with
matrix-nginx-proxy. Previously, running `--tags=setup-nginx-proxy` would
get rid of the Telegram endpoint configuration for the same reason.
Not anymore.
With most people on Synapse v0.99+ and Synapse v1.0 now available,
we should no longer try to be backward compatible with Synapse 0.34,
because this just complicates the instructions for no good reason.
Changes to the original are:
- it tells people to stop and disable services, so that:
- services won't be running while you are copying files
- services won't accidentally start again later
- it does the file-copying in 1 step
- it does copying before running `--tags=setup-all`, so that existing files (SSL certificates, etc.) can be reused. Otherwise, the playbook starts from a blank slate, retrieves them anew, generates new signing keys anew, etc. Only to have those replaced by your own old backup later.
- it mentions DNS changes
- combines `--tags=setup-all,start` into a single step, thanks to the files being already copied
Using `|to_json` on a string is expected to correctly wrap it in quotes (e.g. `"4"`).
Wrapping it explicitly in double-quotes results in undesirable double-quoting (`""4""`).
We do use some `:latest` images by default for the following services:
- matrix-dimension
- Goofys (in the matrix-synapse role)
- matrix-bridge-appservice-irc
- matrix-bridge-appservice-discord
- matrix-bridge-mautrix-facebook
- matrix-bridge-mautrix-whatsapp
It's terribly unfortunate that those software projects don't release
anything other than `:latest`, but that's how it is for now.
Updating that software requires that users manually do `docker pull`
on the server. The playbook didn't force-repull images that it already
had.
With this patch, it starts doing so. Any image tagged `:latest` will be
force re-pulled by the playbook every time it's executed.
It should be noted that even though we ask the `docker_image` module to
force-pull, it only reports "changed" when it actually pulls something
new. This is nice, because it lets people know exactly when something
gets updated, as opposed to giving the indication that it's always
updating the images (even though it isn't).
Previously, it only mentioned exposing for psql-usage purposes.
Realistically, it can be used for much more. Especially given that
psql can be easily accessed via our matrix-postgres-cli script,
without exposing the container port.
We log to journald anyway. There's no need for double-logging.
It should not that matrix-synapse logs to journald and to files,
but that's likely to change in the future as well.
Because Synapse's logs are insanely verbose right now (and may get
dropped by journald), it's more reliable to have file-logging too.
As Synapse matures and gets more stable, logging should hopefully
get less, we should be able to only use journald and stop writing to
files for it as well.
Using a separate directory allows easier backups
(only need to back up the Ansible playbook configuration and the
bridge's `./data` directory).
The playbook takes care of migrating an existing database file
from the base directory into the `./data` directory.
In the future, we can also mount the configuration read-only,
to ensure the bridge won't touch it.
For now, mautrix-facebook is keen on rebuilding the `config.yaml`
file on startup though, so this will have to wait.
Related to #193, but for the Facebook bridge.
(other bridges can be changed to do the same later).
This patch makes the bridge configuration entirely managed by the
Ansible playbook. The bridge's `config.yaml` and `registration.yaml`
configuration files are regenerated every time the playbook runs.
This allows us to apply updates to those files and to avoid
people having to manage the configuration files manually on the server.
-------------------------------------------------------------
A deficiency of the current approach to dumping YAML configuration in
`config.yaml` is that we strip all comments from it.
Later on, when the bridge actually starts, it will load and redump
(this time with comments), which will make the `config.yaml` file
change.
Subsequent playbook runs will report "changed" for the
"Ensure mautrix-facebook config.yaml installed" task, which is a little
strange.
We might wish to improve this in the future, if possible.
Still, it's better to have a (usually) somewhat meaningless "changed"
task than to what we had -- never rebuilding the configuration.
Bridges start matrix-synapse.service as a dependency, but
Synapse is sometimes slow to start, while bridges are quick to
hit it and die (if unavailable).
They'll auto-restart later, but .. this still breaks `--tags=start`,
which doesn't wait long enough for such a restart to happen.
This attempts to slow down bridge startup enough to ensure Synapse
is up and no failures happen at all.
Attempt to fix#192 (Github Issue), potential regression since
70487061f4.
Serializing as JSON/YAML explicitly is much better than relying on
magic (well, Python serialization being valid YAML..).
It seems like Python may prefix strings with `u` sometimes (Python 3?),
which causes Python serialization to not be compatible with YAML.
This doesn't replace all usage of `-v`, but it's a start.
People sometimes troubleshoot by deleting files (especially bridge
config files). Restarting Synapse with a missing registration.yaml file
for a given bridge, causes the `-v
/something/registration.yaml:/something/registration.yaml:ro` option
to force-create `/something/registration.yaml` as a directory.
When a path that's provided to the `-v` option is missing, Docker
auto-creates that path as a directory.
This causes more breakage and confusion later on.
We'd rather fail, instead of magically creating directories.
Using `--mount`, instead of `-v` is the solution to this.
From Docker's documentation:
> When you use --mount with type=bind, the host-path must refer to an existing path on the host.
> The path will not be created for you and the service will fail with an error if the path does not exist.
Related to #189 (Github Issue).
People had proxying problems if:
- they used the whole playbook (including the `matrix-nginx-proxy` role)
- and they were disabling the proxy (`matrix_nginx_proxy_enabled: false`)
- and they were proxying with their own nginx server
For them,
`matrix_nginx_proxy_proxy_matrix_additional_server_configuration_blocks`
would not be modified to inject the necessary proxying configuration.
While using certbot means we'll have both files retrieved,
it's actually the fullchain.pem file that we use in nginx configuration.
Using that one for the check makes more sense.
Reasoning is the same as for matrix-org/synapse#5023.
For us, the journal used to contain `docker` for all services, which
is not very helpful when looking at them all together (`journalctl -f`).
The goal is to move each bridge into its own separate role.
This commit starts off the work on this with 2 bridges:
- mautrix-telegram
- mautrix-whatsapp
Each bridge's role (including these 2) is meant to:
- depend only on the matrix-base role
- integrate nicely with the matrix-synapse role (if available)
- integrate nicely with the matrix-nginx-proxy role (if available and if
required). mautrix-telegram bridge benefits from integrating with
it.
- not break if matrix-synapse or matrix-nginx-proxy are not used at all
This has been provoked by #174 (Github Issue).
The bit about the matrix-make-user-admin script was messed up (it wasn't actually a code block so the "<username>" was hidden). For me at least it seems like the ``` syntax is much harder to accidentally mess up.
As discussed in #151 (Github Pull Request), it's
a good idea to not selectively apply casting, but to do it in all
cases involving arithmetic operations.
This reverts commit 3387035400.
Enabling `jinja2_native` does help with the issue it is trying to
address - #151 (Github Pull Request), but it introduces a regression
when generating templates.
An example is
`roles/matrix-nginx-proxy/templates/nginx/conf.d/matrix-riot-web.conf.j2`,
which yields a strange resulting value of:
```
location /.well-known/acme-challenge {
resolver 127.0.0.11 valid=5s;
set $backend "matrix-certbot:8080";
proxy_pass http://$backend;
resolver 127.0.0.11 valid=5s;
set $backend "matrix-certbot:8080";
proxy_pass http://$backend;
}
```
For whatever reason (still to be investigated), the `if` block's
contents seem to have been outputted twice.
Reverting until this is resolved.
Until then, #151 would rely on the workaround and not on `jinja2_native`.
Helps with #151 (Github Pull Request), but only for Ansible >= 2.7
and when Jinja >= 2.10 is in use.
For other version combinations we still need the workaround proposed
in the pull rqeuest.
Previously, we'd show an error like this:
{"changed": false, "item": null, "msg": "Detected an undefined required variable"}
.. which didn't mention the variable name
(`matrix_ssl_lets_encrypt_support_email`).
Looks like we may not have to do this,
since 1.4.2 fixes edge cases for people who used the broken
1.4.0 release.
We jumped straight to 1.4.1, so maybe we're okay.
Still, upgrading anyway, just in case.
Use an int conversion in the computation of the value of
matrix_nginx_proxy_tmp_directory_size_mb, to have the integer value
multiplied by 50 instead of having the string repeated 50 times.
It doesn't hurt to attempt renewal more frequently, as it only does
real work if it's actually necessary.
Reloading, we postpone some more, because certbot adds some random delay
(between 1 and 8 * 60 seconds) when renewing. We want to ensure
we reload at least 8 minutes later, which wasn't the case.
To make it even safer (in case future certbot versions use a longer
delay), we reload a whole hour later. We're in no rush to start using
the new certificates anyway, especially given that we attempt renewal
often.
Somewhat fixes#146 (Github Issue)
The code used to check for a `homeserver.yaml` file and generate
a configuration (+ key) only if such a configuration file didn't exist.
Certain rare cases (setting up with one server name and then
changing to another) lead to `homeserver.yaml` being there,
but a `matrix.DOMAIN.signing.key` file missing (because the domain
changed).
A new signing key file would never get generated, because `homeserver.yaml`'s
existence used to be (incorrectly) satisfactory for us.
From now on, we don't mix things up like that.
We don't care about `homeserver.yaml` anymore, but rather
about the actual signing key.
The rest of the configuration (`homeserver.yaml` and
`matrix.DOMAIN.log.config`) is rebuilt by us in any case, so whether
it exists or not is irrelevant and doesn't need checking.
- matrix_enable_room_list_search - Controls whether searching the public room list is enabled.
- matrix_alias_creation_rules - Controls who's allowed to create aliases on this server.
- matrix_room_list_publication_rules - Controls who can publish and which rooms can be published in the public room list.
Putting a lot of comments inbetween `[matrix-servers]` and the example
host line may make someone decide to clean up the comment
and accidentally skip-over the `[matrix-servers]` part.
`{% matrix_s3_media_store_custom_endpoint_enabled %}` should have
been `{% if matrix_s3_media_store_custom_endpoint_enabled %}` instead.
Related to #132 (Github Pull Request).
In most cases, there's not really a need to touch the system
firewall, as Docker manages iptables by itself
(see https://docs.docker.com/network/iptables/).
All ports exposed by Docker containers are automatically whitelisted
in iptables and wired to the correct container.
This made installing firewalld and whitelisting ports pointless,
as far as this playbook's services are concerned.
People that wish to install firewalld (for other reasons), can do so
manually from now on.
This is inspired by and fixes#97 (Github Issue).
Fixes#129 (Github Issue).
Unfortunately, we rely on `service_facts`, which is only available
in Ansible >= 2.5.
There's little reason to stick to an old version such as Ansible 2.4:
- some time has passed since we've raised version requirements - it's
time to move into the future (a little bit)
- we've recently (in 82b4640072) improved the way one can run
Ansible in a Docker container
From now on, Ansible >= 2.5 is required.
Inspired by #128 (Github Issue), we've created a new Docker image
to replace https://hub.docker.com/r/qmxme/ansible
Adding dnspython or dig to `qmxme/ansible` doesn't seem like a good
idea (that might be accepted by them), given that it's specific to our
use case. That's why we'll be maintaining our own image from now on.
When using Let's Encrypt SSL certificates, a cronjob is set up to
automatically renew them. Though it does require a `cron`-compatible
program on the server.
This fixes the error that is caused by the `/etc/cron.d` directory
not existing and the `ansible-cron` module trying to write out a
file there -- without checking if the directory exists first.
By default, `--tags=self-check` no longer validates certificates
when `matrix_ssl_retrieval_method` is set to `self-signed`.
Besides this default, people can also enable/disable validation using the
individual role variables manually.
Fixes#124 (Github Issue)
Most (all?) of our Matrix services are running in the `matrix` network,
so they were safe -- not accessible from Coturn to begin with.
Isolating Coturn into its own network is a security improvement
for people who were starting other services in the default
Docker network. Those services were potentially reachable over the
private Docker network from Coturn.
Discussed in #120 (Github Pull Request)
This is more explicit than hiding it in the role defaults.
People who reuse the roles in their own playbook (and not only) may
incorrectly define `ansible_host` to be a hostname or some local address.
Making it more explicit is more likely to prevent such mistakes.
Currently the nginx reload cron fails on Debian 9 because the path to
systemctl is /bin/systemctl rather than /usr/bin/systemctl.
CentOS 7 places systemctl in both /bin and /usr/bin, so we can just use
/bin/systemctl as the full path.
This allows overriding the default value for `include_content`. Setting
this to false allows homeserver admins to ensure that message content
isn't sent in the clear through third party servers.
`matrix_nginx_proxy_data_path` has always served as a base path,
so we're renaming it to reflect that.
Along with this, we're also introducing a new "data path" variable
(`matrix_nginx_proxy_data_path`), which is really a data path this time.
It's used for storing additional, non-configuration, files related to
matrix-nginx-proxy.
It's been reported that YAML parsing errors
would occur on certain Ansible/Python combinations for some reason.
It appears that a bare `{{ matrix_dimension_admins }}` would sometimes
yield things like `[u'@user:domain.com', ..]` (note the `u` string prefix).
To prevent such problems, we now explicitly serialize with `|to_json`.
It didn't mention `matrix_appservice_discord_client_id` and
`matrix_appservice_discord_bot_token`, which makes it hard for
beginners.
Related to #105 (Github Pull Request).
The Server spec says that redirects should be followed for
`/.well-known/matrix/server`. So we follow them.
The Client-Server specs doesn't mention redirects, so we don't
follow redirects there.
Using `docker_container` with a `cap_drop` argument requires
Ansible >=2.7.
We want to support older versions too (2.4), so we either need to
stop invoking it with `cap_drop` (insecure), or just stop using
the module altogether.
Since it was suffering from other bugs too (not deleting containers
on failure), we've decided to remove `docker_container` usage completely.
Some resources shouldn't be cached right now,
as per https://github.com/vector-im/riot-web/pull/8702
(note all of the suggestions from that pull request were applied,
because some of them do not seem relevant - no such files)
Fixes#98 (Github Issue)
`matrix_synapse_no_tls` is now implicit, so we've gotten rid of it.
The `homeserver.yaml.j2` template has been synchronized with the
configuration generated by Synapse v0.99.1 (some new options
are present, etc.)
We had something like that on the Server Delegation how-to page,
but it's better if we have it on the SSL certificates page.
Relocated there and improved linking.
Fixes#94 (Github Issue)
For consistency with all our other listeners,
we make this one bind on the `::` address too
(both IPv4 and IPv6).
Additional details are in #91 (Github Pull Request).
People who wish to rely on SRV records can prevent
the `/.well-known/matrix/server` file from being generated
(and thus, served.. which causes trouble).
If someone decides to not use `/.well-known/matrix/server` and only
relies on SRV records, then they would need to serve tcp/8448 using
a certificate for the base domain (not for the matrix) domain.
Until now, they could do that by giving the certificate to Synapse
and setting it terminate TLS. That makes swapping certificates
more annoying (Synapse requires a restart to re-read certificates),
so it's better if we can support it via matrix-nginx-proxy.
Mounting certificates (or any other file) into the matrix-nginx-proxy container
can be done with `matrix_nginx_proxy_container_additional_volumes`,
introduced in 96afbbb5a.
Certain use-cases may require that people mount additional files
into the matrix-nginx-proxy container. Similarly to how we do it
for Synapse, we are introducing a new variable that makes this
possible (`matrix_nginx_proxy_container_additional_volumes`).
This makes the htpasswd file for Synapse Metrics (introduced in #86,
Github Pull Request) to also perform mounting using this new mechanism.
Hopefully, for such an "extension", keeping htpasswd file-creation and
volume definition in the same place (the tasks file) is better.
All other major volumes' mounting mechanism remains the same (explicit
mounting).
Continuation of 1f0cc92b33.
As an explanation for the problem:
when saying `localhost` on the host, it sometimes gets resolved to `::1`
and sometimes to `127.0.0.1`. On the unfortunate occassions that
it gets resolved to `::1`, the container won't be able to serve the
request, because Docker containers don't have IPv6 enabled by default.
To avoid this problem, we simply prevent any lookups from happening
and explicitly use `127.0.0.1`.
This reverts commit 0dac5ea508.
Relying on pyOpenSSL is the Ansible way of doing things, but is
impractical and annoying for users.
`openssl` is easily available on most servers, even by default.
We'd better use that.
Seems like we unintentionally removed the mounting of certificates
(the `/matrix-config` mount) as part of splitting the playbook into
roles in 51312b8250.
It appears that those certificates weren't necessary for coturn to
funciton though, so we might just get rid of the configuration as well.
We run containers as a non-root user (no effective capabilities).
Still, if a setuid binary is available in a container image, it could
potentially be used to give the user the default capabilities that the
container was started with. For Docker, the default set currently is:
- "CAP_CHOWN"
- "CAP_DAC_OVERRIDE"
- "CAP_FSETID"
- "CAP_FOWNER"
- "CAP_MKNOD"
- "CAP_NET_RAW"
- "CAP_SETGID"
- "CAP_SETUID"
- "CAP_SETFCAP"
- "CAP_SETPCAP"
- "CAP_NET_BIND_SERVICE"
- "CAP_SYS_CHROOT"
- "CAP_KILL"
- "CAP_AUDIT_WRITE"
We'd rather prevent such a potential escalation by dropping ALL
capabilities.
The problem is nicely explained here: https://github.com/projectatomic/atomic-site/issues/203
This is a known/intentional regression since f92c4d5a27.
The new stance on this is that most people would not have
dnspython, but may have the `dig` tool. There's no good
reason for not increasing our chances of success by trying both
methods (Ansible dig lookup and using the `dig` CLI tool).
Fixes#85 (Github issue).
This makes all containers (except mautrix-telegram and
mautrix-whatsapp), start as a non-root user.
We do this, because we don't trust some of the images.
In any case, we'd rather not trust ALL images and avoid giving
`root` access at all. We can't be sure they would drop privileges
or what they might do before they do it.
Because Postfix doesn't support running as non-root,
it had to be replaced by an Exim mail server.
The matrix-nginx-proxy nginx container image is patched up
(by replacing its main configuration) so that it can work as non-root.
It seems like there's no other good image that we can use and that is up-to-date
(https://hub.docker.com/r/nginxinc/nginx-unprivileged is outdated).
Likewise for riot-web (https://hub.docker.com/r/bubuntux/riot-web/),
we patch it up ourselves when starting (replacing the main nginx
configuration).
Ideally, it would be fixed upstream so we can simplify.
We do match the defaults anyway (by default that is),
but people can customize `matrix_user_uid` and `matrix_user_uid`
and it wouldn't be correct then.
In any case, it's better to be explicit about such an important thing.
If this is a brand new server and Postgres had never started,
detecting it before we even start it is not possible.
This moves the logic, so that it happens later on, when Postgres
would have had the chance to start and possibly initialize
a new empty database.
Fixes#82 (Github issue)
The matrix-nginx-proxy role can now be used independently.
This makes it consistent with all other roles, with
the `matrix-base` role remaining as their only dependency.
Separating matrix-nginx-proxy was relatively straightforward, with
the exception of the Mautrix Telegram reverse-proxying configuration.
Mautrix Telegram, being an extension/bridge, does not feel important enough
to justify its own special handling in matrix-nginx-proxy.
Thus, we've introduced the concept of "additional configuration blocks"
(`matrix_nginx_proxy_proxy_matrix_additional_server_configuration_blocks`),
where any module can register its own custom nginx server blocks.
For such dynamic registration to work, the order of role execution
becomes important. To make it possible for each module participating
in dynamic registration to verify that the order of execution is
correct, we've also introduced a `matrix_nginx_proxy_role_executed`
variable.
It should be noted that this doesn't make the matrix-synapse role
dependent on matrix-nginx-proxy. It's optional runtime detection
and registration, and it only happens in the matrix-synapse role
when `matrix_mautrix_telegram_enabled: true`.
With this change, the following roles are now only dependent
on the minimal `matrix-base` role:
- `matrix-corporal`
- `matrix-coturn`
- `matrix-mailer`
- `matrix-mxisd`
- `matrix-postgres`
- `matrix-riot-web`
- `matrix-synapse`
The `matrix-nginx-proxy` role still does too much and remains
dependent on the others.
Wiring up the various (now-independent) roles happens
via a glue variables file (`group_vars/matrix-servers`).
It's triggered for all hosts in the `matrix-servers` group.
According to Ansible's rules of priority, we have the following
chain of inclusion/overriding now:
- role defaults (mostly empty or good for independent usage)
- playbook glue variables (`group_vars/matrix-servers`)
- inventory host variables (`inventory/host_vars/matrix.<your-domain>`)
All roles default to enabling their main component
(e.g. `matrix_mxisd_enabled: true`, `matrix_riot_web_enabled: true`).
Reasoning: if a role is included in a playbook (especially separately,
in another playbook), it should "work" by default.
Our playbook disables some of those if they are not generally useful
(e.g. `matrix_corporal_enabled: false`).
We've previously changed a bunch of lists in `homeserver.yaml.j2`
to be serialized using `|to_nice_yaml`, as that generates a more
readable list in YAML.
`matrix_synapse_federation_domain_whitelist`, however, couldn't have
been changed to that, as it can potentially be an empty list.
We may be able to differentiate between empty and non-empty now
and serialize it accordingly (favoring `|to_nice_yaml` if non-empty),
but it's not important enough to be justified. Thus, always
serializing with `|to_json`.
Fixes#78 (Github issue)
Riot-web parses integrations_widgets_urls as a list, thus causing it to incorrectly think Scalar widgets are non-Scalar and not passing the scalar token
As suggested in #63 (Github issue), splitting the
playbook's logic into multiple roles will be beneficial for
maintainability.
This patch realizes this split. Still, some components
affect others, so the roles are not really independent of one
another. For example:
- disabling mxisd (`matrix_mxisd_enabled: false`), causes Synapse
and riot-web to reconfigure themselves with other (public)
Identity servers.
- enabling matrix-corporal (`matrix_corporal_enabled: true`) affects
how reverse-proxying (by `matrix-nginx-proxy`) is done, in order to
put matrix-corporal's gateway server in front of Synapse
We may be able to move away from such dependencies in the future,
at the expense of a more complicated manual configuration, but
it's probably not worth sacrificing the convenience we have now.
As part of this work, the way we do "start components" has been
redone now to use a loop, as suggested in #65 (Github issue).
This should make restarting faster and more reliable.
This change is provoked by a few different things:
- #54 (Github Pull Request), which rightfully says that we need a
way to support ALL mxisd configuration options easily
- the upcoming mxisd 1.3.0 release, which drops support for
property-style configuration (dot-notation), forcing us to
redo the way we generate the configuration file
With this, mxisd is much more easily configurable now
and much more easily maintaneable by us in the future
(no need to introduce additional playbook variables and logic).
As suggested in #65 (Github issue), this patch switches
cronjob management from using templates to using Ansible's `cron` module.
It also moves the management of the nginx-reload cronjob to `setup_ssl_lets_encrypt.yml`,
which is a more fitting place for it (given that this cronjob is only required when
Let's Encrypt is used).
Pros:
- using a module is more Ansible-ish than templating our own files in
special directories
- more reliable: will fail early (during playbook execution) if `/usr/bin/crontab`
is not available, which is more of a guarantee that cron is working fine
(idea: we should probably install some cron package using the playbook)
Cons:
- invocation schedule is no longer configurable, unless we define individual
variables for everything or do something smart (splitting on ' ', etc.).
Likely not necessary, however.
- requires us to deprecate and clean-up after the old way of managing cronjobs,
because it's not compatible (using the same file as before means appending
additional jobs to it)
This means we no longer have a dependency on the `dig` program,
but we do have a dependency on `dnspython`.
Improves things as suggested in #65 (Github issue).
After having multiple people report issues with retrieving
SSL certificates, we've finally discovered the culprit to be
Ansible 2.5.1 (default and latest version on Ubuntu 18.04 LTS).
As silly as it is, certain distributions ("LTS" even) are 13 bugfix
versions of Ansible behind.
From now on, we try to auto-detect buggy Ansible versions and tell the
user. We also provide some tips for how to upgrade Ansible or
run it from inside a Docker container.
My testing shows that Ansible 2.4.0 and 2.4.6 are OK.
All other intermediate 2.4.x versions haven't been tested, but we
trust they're OK too.
From the 2.5.x releases, only 2.5.0 and 2.5.1 seem to be affected.
Ansible 2.5.2 corrects the problem with `include_tasks` + `with_items`.
This is a simplification and a way to make it consistent with
how we do Postgres imports (see 6d89319822), using
files coming from the server, not from the local machine.
By encouraging people NOT to use local files,
we potentially avoid problems such as #34 (Github issue),
where people would download `media_store` to their Mac's filesystem
and case-sensitivity issues will actually corrupt it.
By not encouraging local files usage, it's less likely that
people would copy (huge) directories to their local machine like that.
This is a simplification and a way to make it consistent with
how we do Postgres imports (see 6d89319822), using
files coming from the server, not from the local machine.
Until now, if the .sql file contained invalid data, psql would
choke on it, but still return an exit code of 0.
This is very misleading.
We need to pass `-v ON_ERROR_STOP=1` to make it exit
with a proper error exit code when failures happen.
We've had that logic in 2 places so far, leading to duplication
and a maintenance burden.
In the future, we'll also have an import-postgres feature,
which will also need Postgres version detection,
leading to more benefit from that logic being reusable.
Fixes#18 (Github issue).
It would probably be better if we serve our own page,
as the Matrix one says:
"To use this server you'll need a Matrix client", which
is true, but we install Riot by default and it'd be better if we mention
that instead.
If uppercase is used, certain tools (like certbot) would cause trouble.
They would retrieve a certificate for the lowercased domain name,
but we'd try to use it from an uppercase-named directory, which will
fail.
Besides certbot, we may experience other trouble too.
(it hasn't been investigated how far the breakage goes).
To fix it all, we lowercase `host_specific_hostname_identity` by default,
which takes care of the general use-case (people only setting that
and relying on us to build the other domain names - `hostname_matrix`
and `hostname_riot`).
For others, who decide to override these other variables directly
(and who may work around us and introduce uppercase there directly),
we also have the sanity-check tool warn if uppercase is detected
in any of the final domains.
Adds support for managing certificates manually and for
having the playbook generate self-signed certificates for you.
With this, Let's Encrypt usage is no longer required.
Fixes Github issue #50.
This is in line with what the recommendation is for matrix-corporal.
A value higher than 30 seconds is required to satisfy Riot
(and other clients') default long-polling behavior.
It looks like SELinux can be left running without any (so far) negative
effects on our Matrix services.
There's no need to use `:z` or `:Z` options when mounting volumes either.
This means that files we create are labeled with a default context
(which may not be ideal if we only want them used from containers),
but it's compatible and doesn't cause issues.
Relabelling files is probably something we wish to stay away from,
especially for things like the media store, which contains lots of
files and is possibly on a fuse-mounted (S3/goofys) filesystem.
The new image is built in a much better way (2-stage build)
and is 10x smaller.
In terms of Goofys version recency, it's about the same..
Both images (and others alike) seem to not use version tags,
but rather some `:latest` (master), with ewoutp/goofys being a bit
more recent than clodproto/goofys.
Not using version tags is good (in this case),
because the last Goofys release seems to be from about a year ago
and there had been a bunch of bugfixes afterwards.
This is described in Github issue #58.
Until now, we had the variable, but if you redefined it, you'd run
into multiple problems:
- we actually always mounted some "storage" directory to the Synapse
container. So if your media store is not there, you're out of luck
- homeserver.yaml always hardcoded the path to the media store,
as a directory called "media-store" inside the storage directory.
Relocating to outside the storage directory was out of the question.
Moreover, even if you had simply renamed the media store directory
(e.g. "media-store" -> "media_store"), it would have also caused trouble.
With this patch, we mount the media store's parent to the Synapse container.
This way, we don't care where the media store is (inside storage or
not). We also don't assume (anymore) that the final part of the path
is called "media-store" -- anything can be used.
The "storage" directory and variable (`matrix_synapse_storage_path`)
still remain for compatibility purposes. People who were previously
overriding `matrix_synapse_storage_path` can continue doing so
and their media store will be at the same place.
The playbook no longer explicitly creates the `matrix_synapse_storage_path` directory
though. It's not necessary. If the media store is specified to be within it, it will
get created when the media store directory is created by the playbook.
Previously, it was more necessary to have it
(because we had a dependency between matrix-synapse and matrix-nginx-proxy)..
But nowadays, it can be removed without negative side effects.
Restarting matrix-nginx-proxy is especially bad when the proxy is not installed at all.
Relay hostnames that have MX records are looked up by postfix
and the MX record's payload is used instead.
This special behavior may be undesirable, so we make sure to
point it out.
mxisd supports several identity stores. Add support to configure two of them:
* synapseSql (storing identities directly in Synapse's database)
* LDAP
This removed the need to copy `mxisd.yaml.j2` to the inventory in case one wants
to use LDAP as identity store. Note that the previous solution (copying
`mxisd.yaml.j2` was poor because of two reasons:
* The copy remains outdated in case the original is updated in future versions
of this repo.
* The role's configuration should be in one place (configured only through role
variables) instead of in multiple.
Configuring more identity stores through role variables can be supported in the
future.
This is provoked by Github issue #46.
No client had made use of the well-known mechanism
so far, so the set up performed by this playbook was not tested
and turned out to be a little deficient.
Even though /.well-known/matrix/client is usually requested with a
simple request (no preflight), it's still considered cross-origin
and [CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS)
applies. Thus, the file always needs to be served with the appropriate
`Access-Control-Allow-Origin` header.
Github issue #46 attempts to fix it at the "reverse-proxying" layer,
which may work, but would need to be done for every server.
It's better if it's done "upstream", so that all reverse-proxy
configurations can benefit.
We've had some people get confused into installing
Matrix Corporal and having pain with that.
With this documentation change, we try to make it clearer
that it's an advanced feature not to be touched unless
you know what you're doing.
On a similar note, we also make sure other things are properly
labeled as "(optional)" and/or "(advanced)".
Trying to:
- stay closer to naming in Synapse (autojoin -> auto_join)
- not create new variable namespaces (`matrix_homeserver_`),
when existing ones (`matrix_synapse_`) are more suitable
- allow `null` (`~`) values for `matrix_riot_web_welcome_user_id`
- render things like `auto_join_rooms` in `homeserver.yaml` more prettily
- fix breakage in `config.json` where `matrix_riot_web_roomdir_servers`
was rendered as YAML and not as JSON
- simplify code (especially in riot-web's `config.json`), which used
`if` statements that could have been omitted
- avoid changing comments in `homeserver.yaml` which are not ours,
so that we can keep closer to the configuration file generated by upstream
Pretty much all variables live in their own `matrix_<whatever>`
prefix now and are grouped closer together in the default
variables file (`roles/matrix-server/defaults/main.yml`).
It should be `/bin/mkdir` and `/bin/chown` on Ubuntu 18.04 for example.
Still, it doesn't seem like we need to create and chown these
directories at all, since the playbook takes care of creating them
and setting appropriate permission by itself.
If a network like `matrix-whatever` already exists for some reason,
the `docker_network` module would not create our `matrix` network.
Working around it by avoiding `docker_network` and doing it manually.
Fixes Github issue #12
`--log-driver=none` is used for all Docker containers now.
All these containers are started through systemd anyway and get logged in journald,
so there's no need for Docker to be logging the same thing using the default `json-file` driver.
Doing that was growing `/var/lib/docker/containers/..` infinitely until service/container restart.
As a result of this, things like `docker logs matrix-synapse` won't work anymore.
`journalctl -u matrix-synapse` is how one can see the logs.
If the playbook were to run with `--tags=setup-nginx-proxy`,
it wouldn't go into `setup_corporal.yml`, which meant it wouldn't
perform a bunch of `set_fact` calls which override important
nginx proxy configuration.
We run these variable overrides on each call now (tagged with `always`)
to avoid such problems in the future.
This disables federation on the 80 port, as it's
not necessary. We also disable the old Angular webclient.
For the federation port (8448), we disable the client APIs
as those are not necessary. Those can even cause trouble
if one doesn't know about them and thinks that guarding the client
APIs at the 80 port is enough.
Moving away from using the default bridge network to using our own.
This isolates our services from other Docker containers running
on the default network on the same host.
The benefits are that:
- isolation is a little better - we no longer share a default
bridge network with any other containers that might be running on the host
- there are no longer hard dependencies - we do service discovery
by DNS name, and not via explicit `--link` usage during container start,
so containers can start out of order and fail without bringing down others
with them
(`matrix-nginx-proxy` can continue running, even if one of the other services dies)
In the future, when other services get introduced,
the increased resilience and simplicity will help as well.
Until now, we were starting from a fresh configuration, as generated
by Synapse and manipulating it with regex and line replacements,
until we made it work.
This is more fragile and less predictable, so we're moving to a static
configuration file generated from a Jinja template.
The upside is that configuration will be stable and predictable.
The downside of this new approach is that any manual configuration changes
after the playbook is done, will be thrown away on future playbook
invocations.
There are 2 ways to work around the need for manual configuration
changes though:
- making them part of this playbook and its default template
configuration files (which benefits everyone)
- going your own way for a given host and overriding the template files
that gets used (that is, the
`matrix_synapse_template_synapse_homeserver` or
`matrix_synapse_template_synapse_log` variables)
This reverts commit 8d774db3bc.
Docker is released in the Docker CE stable repository now.
Additionally, it's version 18.03, which doesn't suffer
any of the problems we've observed with 18.05 (edge/nightly).
This playbook does not set up guest access in Synapse anyway,
so until the need comes (or someone asks for it), guest access
is removed from riot-web's UI too.
As for supporting custom URLs, this is also not something
that seems like it'd be useful to most deployments.
We have 2 blockers that prevent us from adding support:
- the Docker CE repository does not publish a `docker-ce` package
in the `stable` channel. It's still in `edge`
(can be worked around by using `edge`, but we'd better not)
- Docker bind propagation has troubles on Docker CE 18.05,
which breaks matrix-synapse.service from starting, as it wants to do
a `:slave` mount. See https://github.com/moby/moby/issues/37032
Since cbee084ac1, this playbook supports Postgres 10.x,
but keeps existing Postgres-9.x installs on 9.x.
This playbook can now also be ran with `--tags=upgrade-postgres`
to make it upgrade from Postgres 9.x to 10.x (or other versions
in the future).
This playbook just tries to avoid trying to setup a Postgres 10
database with existing 9.x files, as that makes Postgres complain.
Due to this, existing installs (still on 9.x) are detected
and left on Postgres 9.x.
They need to be upgraded to Postgres 10.x manually.
Switching from from avhost/docker-matrix (silviof/docker-matrix)
to matrixdotorg/synapse.
The avhost/docker-matrix (silviof/docker-matrix) image used to bundle
in the coturn STUN/TURN server, so as part of the move,
we're separating this to a separately-ran service
(matrix-coturn.service, powered by instrumentisto/coturn-docker-image)
A `.log.config` file may be generated with a different
level of indentation depending on which (Docker image, etc.)
generates it.
With this patch, we tolerate different levels of indentation
(2 spaces, 4 spaces, etc.) and don't break the configuration.
When using matrix-nginx-proxy, the file permissions are organized
in a way that matrix-nginx-proxy could read the challenge files
produced by acmetool.
However, when another own/external webserver was used (like nginx
with our generated sample configuration), this could not work.
From on we're proxying the HTTP requests to port :402 in such a case,
which fixes the problem.
The matrix-nginx-proxy was reloaded on the 3rd day of the month (`15 4 3 * *`),
which makes no sense - it's too infrequently.
It's in line with the renewal time now (+5 minutes).
The non-working script is supposed to be fixed
by https://github.com/matrix-org/synapse/pull/2375
To have it work, we'd need an updated Docker image
of `silviof/matrix-riot-docker:latest`, which is not yet available
at the time of this commit.
Still, the previous patched synapse_port_db didn't work well either,
so it's not like we're regressing much by getting rid of it.
As described here (
https://github.com/matrix-org/synapse/issues/2438#issuecomment-327424711
), using own SSL certificates for the federation port is more fragile,
as renewing them could cause federation outages.
The recommended setup is to use the self-signed certificates generated
by Synapse.
On the 443 port (matrix-nginx-proxy) side, we still use the Let's Encrypt
certificates, which ensures API consumers work without having to trust
"our own CA".
Having done this, we also don't need to ever restart Synapse anymore,
as no new SSL certificates need to be applied there.
It's just matrix-nginx-proxy that needs to be restarted, and it doesn't
even need a full restart as an "nginx reload" does the job of swithing
to the new SSL certificates.
Moving keeps everything in the /matrix directory, so that we
wouldn't contaminate anything else on the system or risk
clashing with something else.
Also retrieving certificates separately for the Riot and Matrix domains,
which should help in multiple ways:
- allows them to be very different (completely separate base domain..)
- allows for Riot to be disabled for the playbook some time later
and still have the code not break
Let's let the admin set them as they wish.
We don't care what they are anyway.
If other things run on the same server,
it's also better not to hijack these for our
own purposes, especially when we don't need to.
The timedatectl call also seems to fail on Ubuntu 17.04
for some reason (missing timezones information file?).
The goal is to allow these to be on separate partitions
(including remote ones in the future).
Because the `silviof/docker-matrix` image chowns
everything to MATRIX_UID:MATRIX_GID on startup,
we definitely don't want to include `media_store` in it.
If it's on a remote FS, it would cause a slow startup.
Also, adding some safety checks to the "import media store"
task, after passing a wrong path to it on multiple occassions and
wondering what's wrong.
Also, making logging configurable. The default of keeping 10x100MB
log files is likely excessive and people may want to change that.
Port 8008 is forwarded in our case, so unless we adjust
`x_forwaded` for it, Docker's local network IPs are
logged/displayed for devices.
The TLS port (8448) is not proxied in our setup,
so its `x_forwarded` setting remains `false`.
Otherwise certains values in the config file,
such as `macaroon_secret_key`, would be regenerated,
which is not something that we want.
If `macaroon_secret_key` is regenerated, all users'
auth tokens will become invalid (effectively logging out
all users).
It was trying to omit the `-a` flag, but that wasn't enough,
because the underlying `register_new_matrix_user` command
prompts interactively if it doesn't see the `-a` flag
(it doesn't default to non-admin).
We need to answer such interactive prompts.
Some CentOS 7 hosts may not have firewalld installed.
We shouldn't expect it to be, but should ensure by ourselves that it is.
Docker likes to mess around with iptables forwarding rules,
so it ought to start after firewalld.
matrix-nginx-proxy will be occupying port 80 soon,
so that we can be more user-friendly and have
http->https forwarding for the Riot hostname.
During the playbook run, acmetool also expects to use
port 80 for domain verification.
During an initial playbook run, this wouldn't cause trouble
because matrix-nginx-proxy is not installed yet.
However, on subsequent playbook runs, it would cause trouble.
This ensures that if matrix-nginx-proxy is available
and running, it would be stopped before running acmetool
and started right after.
[](https://matrix.to/#/#matrix-docker-ansible-deploy:devture.com) [](https://liberapay.com/s.pantaleev/donate)
# Matrix (An open network for secure, decentralized communication) server setup using Ansible and Docker
## Purpose
This [Ansible](https://www.ansible.com/) playbook is meant to help you run your own [Matrix](http://matrix.org/) homeserver, along with the [various services](#supported-services) related to that.
That is, it lets you join the Matrix network using your own `@<username>:<your-domain>` identifier, all hosted on your own server (see [prerequisites](docs/prerequisites.md)).
We run all services in [Docker](https://www.docker.com/) containers (see [the container images we use](docs/container-images.md)), which lets us have a predictable and up-to-date setup, across multiple supported distros (see [prerequisites](docs/prerequisites.md)) and [architectures](docs/alternative-architectures.md) (x86/amd64 being recommended).
[Installation](docs/README.md) (upgrades) and some maintenance tasks are automated using [Ansible](https://www.ansible.com/) (see [our Ansible guide](docs/ansible.md)).
## Supported services
Using this playbook, you can get the following services configured on your server:
- (optional, default) a [Synapse](https://github.com/matrix-org/synapse) homeserver - storing your data and managing your presence in the [Matrix](http://matrix.org/) network
- (optional) [Amazon S3](https://aws.amazon.com/s3/) storage for Synapse's content repository (`media_store`) files using [Goofys](https://github.com/kahing/goofys)
- (optional, default) [PostgreSQL](https://www.postgresql.org/) database for Synapse. [Using an external PostgreSQL server](docs/configuring-playbook-external-postgres.md) is also possible.
- (optional, default) a [coturn](https://github.com/coturn/coturn) STUN/TURN server for WebRTC audio/video calls
- (optional, default) free [Let's Encrypt](https://letsencrypt.org/) SSL certificate, which secures the connection to the Synapse server and the Element web UI
- (optional, default) an [Element](https://app.element.io/) ([formerly Riot](https://element.io/previously-riot)) web UI, which is configured to connect to your own Synapse server by default
- (optional, default) a [ma1sd](https://github.com/ma1uta/ma1sd) Matrix Identity server
- (optional, default) an [Exim](https://www.exim.org/) mail server, through which all Matrix services send outgoing email (can be configured to relay through another SMTP server)
- (optional, default) an [nginx](http://nginx.org/) web server, listening on ports 80 and 443 - standing in front of all the other services. Using your own webserver [is possible](docs/configuring-playbook-own-webserver.md)
- (optional, advanced) the [matrix-synapse-rest-auth](https://github.com/ma1uta/matrix-synapse-rest-password-provider) REST authentication password provider module
- (optional, advanced) the [matrix-synapse-shared-secret-auth](https://github.com/devture/matrix-synapse-shared-secret-auth) password provider module
- (optional, advanced) the [matrix-synapse-ldap3](https://github.com/matrix-org/matrix-synapse-ldap3) LDAP Auth password provider module
- (optional, advanced) the [synapse-simple-antispam](https://github.com/t2bot/synapse-simple-antispam) spam checker module
- (optional, advanced) the [Matrix Corporal](https://github.com/devture/matrix-corporal) reconciliator and gateway for a managed Matrix server
- (optional) the [mautrix-telegram](https://github.com/mautrix/telegram) bridge for bridging your Matrix server to [Telegram](https://telegram.org/)
- (optional) the [mautrix-whatsapp](https://github.com/mautrix/whatsapp) bridge for bridging your Matrix server to [WhatsApp](https://www.whatsapp.com/)
- (optional) the [mautrix-facebook](https://github.com/mautrix/facebook) bridge for bridging your Matrix server to [Facebook](https://facebook.com/)
- (optional) the [mautrix-hangouts](https://github.com/mautrix/hangouts) bridge for bridging your Matrix server to [Google Hangouts](https://en.wikipedia.org/wiki/Google_Hangouts)
- (optional) the [mautrix-googlechat](https://github.com/mautrix/googlechat) bridge for bridging your Matrix server to [Google Chat](https://en.wikipedia.org/wiki/Google_Chat)
- (optional) the [mautrix-instagram](https://github.com/mautrix/instagram) bridge for bridging your Matrix server to [Instagram](https://instagram.com/)
- (optional) the [mautrix-signal](https://github.com/mautrix/signal) bridge for bridging your Matrix server to [Signal](https://www.signal.org/)
- (optional) the [beeper-linkedin](https://gitlab.com/beeper/linkedin) bridge for bridging your Matrix server to [LinkedIn](https://www.linkedin.com/)
- (optional) the [matrix-appservice-irc](https://github.com/matrix-org/matrix-appservice-irc) bridge for bridging your Matrix server to [IRC](https://wikipedia.org/wiki/Internet_Relay_Chat)
- (optional) the [matrix-appservice-discord](https://github.com/Half-Shot/matrix-appservice-discord) bridge for bridging your Matrix server to [Discord](https://discordapp.com/)
- (optional) the [matrix-appservice-slack](https://github.com/matrix-org/matrix-appservice-slack) bridge for bridging your Matrix server to [Slack](https://slack.com/)
- (optional) the [matrix-appservice-webhooks](https://github.com/turt2live/matrix-appservice-webhooks) bridge for slack compatible webhooks ([ConcourseCI](https://concourse-ci.org/), [Slack](https://slack.com/) etc. pp.)
- (optional) the [matrix-sms-bridge](https://github.com/benkuly/matrix-sms-bridge) for bridging your Matrix server to SMS - see [docs/configuring-playbook-bridge-matrix-bridge-sms.md](docs/configuring-playbook-bridge-matrix-bridge-sms.md) for setup documentation
- (optional) the [Heisenbridge](https://github.com/hifi/heisenbridge) for bridging your Matrix server to IRC bouncer-style - see [docs/configuring-playbook-bridge-heisenbridge.md](docs/configuring-playbook-bridge-heisenbridge.md) for setup documentation
- (optional) the [mx-puppet-skype](https://hub.docker.com/r/sorunome/mx-puppet-skype) for bridging your Matrix server to [Skype](https://www.skype.com) - see [docs/configuring-playbook-bridge-mx-puppet-skype.md](docs/configuring-playbook-bridge-mx-puppet-skype.md) for setup documentation
- (optional) the [mx-puppet-slack](https://hub.docker.com/r/sorunome/mx-puppet-slack) for bridging your Matrix server to [Slack](https://slack.com) - see [docs/configuring-playbook-bridge-mx-puppet-slack.md](docs/configuring-playbook-bridge-mx-puppet-slack.md) for setup documentation
- (optional) the [mx-puppet-instagram](https://github.com/Sorunome/mx-puppet-instagram) bridge for Instagram-DMs ([Instagram](https://www.instagram.com/)) - see [docs/configuring-playbook-bridge-mx-puppet-instagram.md](docs/configuring-playbook-bridge-mx-puppet-instagram.md) for setup documentation
- (optional) the [mx-puppet-twitter](https://github.com/Sorunome/mx-puppet-twitter) bridge for Twitter-DMs ([Twitter](https://twitter.com/)) - see [docs/configuring-playbook-bridge-mx-puppet-twitter.md](docs/configuring-playbook-bridge-mx-puppet-twitter.md) for setup documentation
- (optional) the [mx-puppet-discord](https://github.com/matrix-discord/mx-puppet-discord) bridge for [Discord](https://discordapp.com/) - see [docs/configuring-playbook-bridge-mx-puppet-discord.md](docs/configuring-playbook-bridge-mx-puppet-discord.md) for setup documentation
- (optional) the [mx-puppet-groupme](https://gitlab.com/robintown/mx-puppet-groupme) bridge for [GroupMe](https://groupme.com/) - see [docs/configuring-playbook-bridge-mx-puppet-groupme.md](docs/configuring-playbook-bridge-mx-puppet-groupme.md) for setup documentation
- (optional) the [mx-puppet-steam](https://github.com/icewind1991/mx-puppet-steam) bridge for [Steam](https://steamapp.com/) - see [docs/configuring-playbook-bridge-mx-puppet-steam.md](docs/configuring-playbook-bridge-mx-puppet-steam.md) for setup documentation
- (optional) [Email2Matrix](https://github.com/devture/email2matrix) for relaying email messages to Matrix rooms - see [docs/configuring-playbook-email2matrix.md](docs/configuring-playbook-email2matrix.md) for setup documentation
- (optional) [Dimension](https://github.com/turt2live/matrix-dimension), an open source integrations manager for matrix clients - see [docs/configuring-playbook-dimension.md](docs/configuring-playbook-dimension.md) for setup documentation
- (optional) [Etherpad](https://etherpad.org), an open source collaborative text editor - see [docs/configuring-playbook-etherpad.md](docs/configuring-playbook-etherpad.md) for setup documentation
- (optional) [Jitsi](https://jitsi.org/), an open source video-conferencing platform - see [docs/configuring-playbook-jitsi.md](docs/configuring-playbook-jitsi.md) for setup documentation
- (optional) [matrix-reminder-bot](https://github.com/anoadragon453/matrix-reminder-bot) for scheduling one-off & recurring reminders and alarms - see [docs/configuring-playbook-bot-matrix-reminder-bot.md](docs/configuring-playbook-bot-matrix-reminder-bot.md) for setup documentation
- (optional) [Go-NEB](https://github.com/matrix-org/go-neb) multi functional bot written in Go - see [docs/configuring-playbook-bot-go-neb.md](docs/configuring-playbook-bot-go-neb.md) for setup documentation
- (optional) [Mjolnir](https://github.com/matrix-org/mjolnir), a moderation tool for Matrix - see [docs/configuring-playbook-bot-mjolnir.md](docs/configuring-playbook-bot-mjolnir.md) for setup documentation
- (optional) [synapse-admin](https://github.com/Awesome-Technologies/synapse-admin), a web UI tool for administrating users and rooms on your Matrix server - see [docs/configuring-playbook-synapse-admin.md](docs/configuring-playbook-synapse-admin.md) for setup documentation
- (optional) [matrix-registration](https://github.com/ZerataX/matrix-registration), a simple python application to have a token based matrix registration - see [docs/configuring-playbook-matrix-registration.md](docs/configuring-playbook-matrix-registration.md) for setup documentation
- (optional) the [Prometheus](https://prometheus.io) time-series database server, the Prometheus [node-exporter](https://prometheus.io/docs/guides/node-exporter/) host metrics exporter, and the [Grafana](https://grafana.com/) web UI - see [Enabling metrics and graphs (Prometheus, Grafana) for your Matrix server](docs/configuring-playbook-prometheus-grafana.md) for setup documentation
- (optional) the [Sygnal](https://github.com/matrix-org/sygnal) push gateway - see [Setting up the Sygnal push gateway](docs/configuring-playbook-sygnal.md) for setup documentation
- (optional) the [Hydrogen](https://github.com/vector-im/hydrogen-web) web client - see [docs/configuring-playbook-client-hydrogen.md](docs/configuring-playbook-client-hydrogen.md) for setup documentation
Basically, this playbook aims to get you up-and-running with all the necessities around Matrix, without you having to do anything else.
**Note**: the list above is exhaustive. It includes optional or even some advanced components that you will most likely not need.
Sticking with the defaults (which install a subset of the above components) is the best choice, especially for a new installation.
You can always re-run the playbook later to add or remove components.
## Installation
To configure and install Matrix on your own server, follow the [README in the docs/ directory](docs/README.md).
## Changes
This playbook evolves over time, sometimes with backward-incompatible changes.
When updating the playbook, refer to [the changelog](CHANGELOG.md) to catch up with what's new.
- [etke.cc](https://etke.cc) - matrix-docker-ansible-deploy and system stuff "as a service". That service will create your matrix homeserver on your domain and server (doesn't matter if it's cloud provider or on an old laptop in the corner of your room), (optional) maintains it (server's system updates, cleanup, security adjustments, tuning, etc.; matrix homeserver updates & maintenance) and (optional) provide full-featured email service for your domain
- [GoMatrixHosting](https://gomatrixhosting.com) - matrix-docker-ansible-deploy "as a service" with [Ansible AWX](https://github.com/ansible/awx). Members can be assigned a server from DigitalOcean, or they can connect their on-premises server. This AWX system can manage the updates, configuration, import and export, backups, and monitoring on its own. For more information [see our GitLab group](https://gitlab.com/GoMatrixHosting) or come [visit us on Matrix](https://matrix.to/#/#general:gomatrixhosting.com).
- [FAQ](faq.md) - lots of questions and answers. Jump to [Prerequisites](prerequisites.md) to avoid reading too much and to just start a guided installation.
- [Prerequisites](prerequisites.md) - go here to a guided installation using this Ansible playbook
- [Configuring your DNS server](configuring-dns.md)
- [Getting this playbook's source code](getting-the-playbook.md)
- [Configuring the playbook](configuring-playbook.md)
- [Installing](installing.md)
- **Importing data from another server installation**
- [Importing an existing SQLite database (from another Synapse installation)](importing-synapse-sqlite.md) (optional)
- [Importing an existing Postgres database (from another installation)](importing-postgres.md) (optional)
- [Importing `media_store` data files from an existing Synapse installation](importing-synapse-media-store.md) (optional)
As stated in the [Prerequisites](prerequisites.md), currently only `x86_64` is fully supported. However, it is possible to set the target architecture, and some tools can be built on the host or other measures can be used.
To that end add the following variable to your `vars.yaml` file:
Currently supported architectures are the following:
- `amd64` (the default)
- `arm64`
- `arm32`
so for the Raspberry Pi, the following should be in your `vars.yaml` file:
```yaml
matrix_architecture: "arm32"
```
## Implementation details
For `amd64`, prebuilt container images (see the [container images we use](container-images.md)) are used everywhere, because all images are available for this architecture.
For other architectures, components which have a prebuilt image make use of it. If the component is not available for the specific architecture, [self-building](self-building.md) will be used. Not all components support self-building though, so your mileage may vary.
This playbook is meant to be run using [Ansible](https://www.ansible.com/).
Ansible typically runs on your local computer and carries out tasks on a remote server.
If your local computer cannot run Ansible, you can also run Ansible on some server somewhere (including the server you wish to install to).
## Supported Ansible versions
Ansible 2.7.1 or newer is required ([last discussion about Ansible versions](https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/743)).
Note: Ubuntu 20.04 ships with Ansible 2.9.6 which is a buggy version (see this [bug](https://bugs.launchpad.net/ubuntu/+source/ansible/+bug/1880359)), which can't be used in combination with a host running new systemd (more details in [#517](https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/517), [#669](https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/669)). If this problem affects you, you can: avoid running Ubuntu 20.04 on your host; run Ansible from another machine targeting your host; or try to upgrade to a newer Ansible version (see below).
## Checking your Ansible version
In most cases, you won't need to worry about the Ansible version.
The playbook will try to detect it and tell you if you're on an unsupported version.
To manually check which version of Ansible you're on, run: `ansible --version`.
If you're on an old version of Ansible, you should [upgrade Ansible to a newer version](#upgrading-ansible) or [use Ansible via Docker](#using-ansible-via-docker).
## Upgrading Ansible
Depending on your distribution, you may be able to upgrade Ansible in a few different ways:
- by using an additional repository (PPA, etc.), which provides newer Ansible versions. See instructions for [CentOS](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html#installing-ansible-on-rhel-centos-or-fedora), [Debian](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html#installing-ansible-on-debian), or [Ubuntu](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html#installing-ansible-on-ubuntu) on the Ansible website.
- by removing the Ansible package (`yum remove ansible` or `apt-get remove ansible`) and installing via [pip](https://pip.pypa.io/en/stable/installing/) (`pip install ansible`).
If using the `pip` method, do note that the `ansible-playbook` binary may not be on the `$PATH` (https://linuxconfig.org/linux-path-environment-variable), but in some more special location like `/usr/local/bin/ansible-playbook`. You may need to invoke it using the full path.
**Note**: Both of the above methods are a bad way to run system software such as Ansible.
If you find yourself needing to resort to such hacks, please consider reporting a bug to your distribution and/or switching to a sane distribution, which provides up-to-date software.
## Using Ansible via Docker
Alternatively, you can run Ansible on your computer from inside a Docker container (powered by the [devture/ansible](https://hub.docker.com/r/devture/ansible/) Docker image).
Here's a sample command to get you started (run this from the playbook's directory):
```bash
docker run -it --rm \
-w /work \
-v `pwd`:/work \
-v $HOME/.ssh/id_rsa:/root/.ssh/id_rsa:ro \
--entrypoint=/bin/sh \
docker.io/devture/ansible:2.9.14-r0
```
The above command tries to mount an SSH key (`$HOME/.ssh/id_rsa`) into the container (at `/root/.ssh/id_rsa`).
If your SSH key is at a different path (not in `$HOME/.ssh/id_rsa`), adjust that part.
Once you execute the above command, you'll be dropped into a `/work` directory inside a Docker container.
The `/work` directory contains the playbook's code.
You can execute `ansible-playbook` commands as per normal now.
### If you don't use SSH keys for authentication
If you don't use SSH keys for authentication, simply remove that whole line (`-v $HOME/.ssh/id_rsa:/root/.ssh/id_rsa:ro`).
To authenticate at your server using a password, you need to add a package. So, when you are in the shell of the ansible docker container (the previously used `docker run -it ...` command), run:
```bash
apk add sshpass
```
Then, to be asked for the password whenever running an `ansible-playbook` command add `--ask-pass` to the arguments of the command.
An AWX setup for managing multiple Matrix servers.
This section is used in an AWX system that can create and manage multiple [Matrix](http://matrix.org/) servers. You can issue members an AWX login to their own 'organisation', which they can use to manage/configure 1 to N servers.
Members can be assigned a server from Digitalocean, or they can connect their own on-premises server. This script is free to use in a commercial context with the 'MemberPress Plus' and 'WP Oauth Sever' addons. It can also be run in a non-commercial context.
The AWX system is arranged into 'members' each with their own 'subscriptions'. After creating a subscription the user enters the 'provision stage' where they defined the URLs they will use, the servers location and whether or not there's already a website at the base domain. They then proceed onto the 'deploy stage' where they can configure their Matrix server.
This system can manage the updates, configuration, import and export, backups and monitoring on its own. It is an extension of the popular deploy script [spantaleev/matrix-docker-ansible-deploy](https://github.com/spantaleev/matrix-docker-ansible-deploy).
## Other Required Playbooks
The following repositories allow you to copy and use this setup:
[Create AWX System](https://gitlab.com/GoMatrixHosting/create-awx-system) - Creates and configures the AWX system for you.
[Ansible Create Delete Subscription Membership](https://gitlab.com/GoMatrixHosting/ansible-create-delete-subscription-membership) - Used by the AWX system to create memberships and subscriptions. Also includes other administrative playbooks for updates, backups and restoring servers.
[Ansible Provision Server](https://gitlab.com/GoMatrixHosting/ansible-provision-server) - Used by AWX members to perform initial configuration of their DigitalOcean or On-Premises server.
## Does I need an AWX setup to use this? How do I configure it?
Yes, you'll need to configure an AWX instance, the [Create AWX System](https://gitlab.com/GoMatrixHosting/create-awx-system) repository makes it easy to do. Just follow the steps listed in ['/docs/Installation_AWX.md' of that repository](https://gitlab.com/GoMatrixHosting/create-awx-system/-/blob/master/docs/Installation_AWX.md).
For simpler installation steps you can use to get started with this system, check out our minimal installation guide at ['/doc/Installation_Minimal_AWX.md of that repository'](https://gitlab.com/GoMatrixHosting/create-awx-system/-/blob/master/docs/Installation_Minimal_AWX.md).
## Does I need a front-end WordPress site? And a DigitalOcean account?
You do not need a front-end WordPress site or any of the mentioned WordPress plugins to use this setup. It can be run on it's own in a non-commercial context.
You also don't need a DigitalOcean account, but this will limit you to only being able to connect 'On-Premises' servers.
To set up Matrix on your domain, you'd need to do some DNS configuration.
To use an identifier like `@<username>:<your-domain>`, you don't actually need
to install anything on the actual `<your-domain>` server.
You do, however need to instruct the Matrix network that Matrix services for `<your-domain>` are delegated
over to `matrix.<your-domain>`.
As we discuss in [Server Delegation](howto-server-delegation.md), there are 2 different ways to set up such delegation:
- either by serving a `https://<your-domain>/.well-known/matrix/server` file (from the base domain!)
- or by using a `_matrix._tcp` DNS SRV record (don't confuse this with the `_matrix-identity._tcp` SRV record described below)
This playbook mostly discusses the well-known file method, because it's easier to manage with regard to certificates.
If you decide to go with the alternative method ([Server Delegation via a DNS SRV record (advanced)](howto-server-delegation.md#server-delegation-via-a-dns-srv-record-advanced)), please be aware that the general flow that this playbook guides you through may not match what you need to do.
## DNS settings for services enabled by default
| Type | Host | Priority | Weight | Port | Target |
As the table above illustrates, you need to create 2 subdomains (`matrix.<your-domain>` and `element.<your-domain>`) and point both of them to your new server's IP address (DNS `A` record or `CNAME` record is fine).
The `element.<your-domain>` subdomain may be necessary, because this playbook installs the [Element](https://github.com/vector-im/element-web) web client for you.
If you'd rather instruct the playbook not to install Element (`matrix_client_element_enabled: false` when [Configuring the playbook](configuring-playbook.md) later), feel free to skip the `element.<your-domain>` DNS record.
The `dimension.<your-domain>` subdomain may be necessary, because this playbook could install the [Dimension integrations manager](http://dimension.t2bot.io/) for you. Dimension installation is disabled by default, because it's only possible to install it after the other Matrix services are working (see [Setting up Dimension](configuring-playbook-dimension.md) later). If you do not wish to set up Dimension, feel free to skip the `dimension.<your-domain>` DNS record.
The `jitsi.<your-domain>` subdomain may be necessary, because this playbook could install the [Jitsi video-conferencing platform](https://jitsi.org/) for you. Jitsi installation is disabled by default, because it may be heavy and is not a core required component. To learn how to install it, see our [Jitsi](configuring-playbook-jitsi.md) guide. If you do not wish to set up Jitsi, feel free to skip the `jitsi.<your-domain>` DNS record.
The `stats.<your-domain>` subdomain may be necessary, because this playbook could install [Grafana](https://grafana.com/) and setup performance metrics for you. Grafana installation is disabled by default, it is not a core required component. To learn how to install it, see our [metrics and graphs guide](configuring-playbook-prometheus-grafana.md). If you do not wish to set up Grafana, feel free to skip the `stats.<your-domain>` DNS record. It is possible to install Prometheus without installing Grafana, this would also not require the `stats.<your-domain>` subdomain.
The `goneb.<your-domain>` subdomain may be necessary, because this playbook could install the [Go-NEB](https://github.com/matrix-org/go-neb) bot. The installation of Go-NEB is disabled by default, it is not a core required component. To learn how to install it, see our [configuring Go-NEB guide](configuring-playbook-bot-go-neb.md). If you do not wish to set up Go-NEB, feel free to skip the `goneb.<your-domain>` DNS record.
The `sygnal.<your-domain>` subdomain may be necessary, because this playbook could install the [Sygnal](https://github.com/matrix-org/sygnal) push gateway. The installation of Sygnal is disabled by default, it is not a core required component. To learn how to install it, see our [configuring Sygnal guide](configuring-playbook-sygnal.md). If you do not wish to set up Sygnal (you probably don't, unless you're also developing/building your own Matrix apps), feel free to skip the `sygnal.<your-domain>` DNS record.
The `hydrogen.<your-domain>` subdomain may be necessary, because this playbook could install the [Hydrogen](https://github.com/vector-im/hydrogen-web) web client. The installation of Hydrogen is disabled by default, it is not a core required component. To learn how to install it, see our [configuring Hydrogen guide](configuring-playbook-client-hydrogen.md). If you do not wish to set up Hydrogen, feel free to skip the `hydrogen.<your-domain>` DNS record.
## `_matrix-identity._tcp` SRV record setup
To make the [ma1sd](https://github.com/ma1uta/ma1sd) Identity Server (which this playbook installs for you) enable its federation features, set up an SRV record that looks like this:
- Name: `_matrix-identity._tcp` (use this text as-is)
- Content: `10 0 443 matrix.<your-domain>` (replace `<your-domain>` with your own)
This is an optional feature. See [ma1sd's documentation](https://github.com/ma1uta/ma1sd/wiki/mxisd-and-your-privacy#choices-are-never-easy) for information on the privacy implications of setting up this SRV record.
Note: This `_matrix-identity._tcp` SRV record for the identity server is different from the `_matrix._tcp` that can be used for Synapse delegation. See [howto-server-delegation.md](howto-server-delegation.md) for more information about delegation.
When you're done with the DNS configuration and ready to proceed, continue with [Getting the playbook](getting-the-playbook.md).
This playbook sets up services on your Matrix server (`matrix.DOMAIN`).
To have this server officially be responsible for Matrix services for the base domain (`DOMAIN`), you need to set up [Server Delegation](howto-server-delegation.md).
This is normally done by [configuring well-known](configuring-well-known.md) files on the base domain.
People who don't have a separate server to dedicate to the base domain have trouble arranging this.
Usually, there are 2 options:
- either get a separate server for the base domain, just for serving the files necessary for [Server Delegation via a well-known file](howto-server-delegation.md#server-delegation-via-a-well-known-file)
- or, arrange for the Matrix server to serve the base domain. This either involves you [using your own webserver](configuring-playbook-own-webserver.md) or making the integrated webserver (`matrix-nginx-proxy`) serve the base domain for you.
This documentation page tells you how to do the latter. With some easy changes, we make it possible to serve the base domain from the Matrix server via the integrated webserver (`matrix-nginx-proxy`).
Just **adjust your DNS records**, so that your base domain is pointed to the Matrix server's IP address (using a DNS `A` record) **and then use the following configuration**:
- obtain an SSL certificate for the base domain, just like it does for all other domains (see [how we handle SSL certificates](configuring-playbook-ssl-certificates.md))
- serve the `/.well-known/matrix/*` files which are necessary for [Federation Server Discovery](configuring-well-known.md#introduction-to-client-server-discovery) (also see [Server Delegation](howto-server-delegation.md)) and [Client-Server discovery](configuring-well-known.md#introduction-to-client-server-discovery)
- serve a simple homepage at `https://DOMAIN` with content `Hello from DOMAIN` (configurable via the `matrix_nginx_proxy_base_domain_homepage_template` variable). You can also [serve a more complicated static website](#serving-a-static-website-at-the-base-domain).
## Serving a static website at the base domain
By default, when "serving the base domain" is enabled, the playbook hosts a simple `index.html` webpage in `/matrix/nginx-proxy/data/matrix-domain`.
The content of this page is taken from the `matrix_nginx_proxy_base_domain_homepage_template` variable.
If you'd like to host your own static website (more than a single `index.html` page) at the base domain, you can disable the creation of this default `index.html` page like this:
With this configuration, Ansible will no longer mess around with the `/matrix/nginx-proxy/data/matrix-domain/index.html` file.
You are then free to upload any static website files to `/matrix/nginx-proxy/data/matrix-domain` and they will get served at the base domain.
## Serving a more complicated website at the base domain
If you'd like to serve an even more complicated (dynamic) website from the Matrix server, relying on the playbook to serve the base domain is not the best choice.
Instead, we recommend that you switch to [using your own webserver](configuring-playbook-own-webserver.md) (preferrably nginx). You can then make that webserver host anything you wish, and still easily plug in Matrix services into it.
Alternatively, you can use a full-featured client (such as Element) to log in and get the access token from there (note: don't log out from the client as that will invalidate the token), but doing so might lead to decryption problems. That warning comes from [here](https://github.com/matrix-org/go-neb#quick-start).
## Adjusting the playbook configuration
Add the following configuration to your `inventory/host_vars/matrix.DOMAIN/vars.yml` file (adapt to your needs):
```yaml
matrix_bot_go_neb_enabled: true
# You need at least 1 client.
# Use the access token you obtained in the step above.
UserID: "@goneb:{{ matrix_domain }}" # requires a Syncing client
Config:
RealmID: "github_realm"
# Make sure your BASE_URL can be accessed by Github!
- ID: "github_webhook_service"
Type: "github-webhook"
UserID: "@another_goneb:{{ matrix_domain }}"
Config:
RealmID: "github_realm"
ClientUserID: "@YOUR_USER_ID:{{ matrix_domain }}" # needs to be an authenticated user so Go-NEB can create webhooks. Check the UserID field in the github_realm in matrix_bot_go_neb_sessions.
Rooms:
"!someroom:id":
Repos:
"matrix-org/synapse":
Events: ["push", "issues"]
"matrix-org/dendron":
Events: ["pull_request"]
"!anotherroom:id":
Repos:
"matrix-org/synapse":
Events: ["push", "issues"]
"matrix-org/dendron":
Events: ["pull_request"]
- ID: "slackapi_service"
Type: "slackapi"
UserID: "@slackapi:{{ matrix_domain }}"
Config:
Hooks:
"hook1":
RoomID: "!someroom:id"
MessageType: "m.text" # default is m.text
- ID: "alertmanager_service"
Type: "alertmanager"
UserID: "@alertmanager:{{ matrix_domain }}"
Config:
# This is for information purposes only. It should point to Go-NEB path as follows:
# `/services/hooks/<base64 encoded service ID>`
# Where in this case "service ID" is "alertmanager_service"
# Make sure your BASE_URL can be accessed by the Alertmanager instance!
To use the bot, invite it to any existing Matrix room (`/invite @whatever_you_chose:DOMAIN` where `YOUR_DOMAIN` is your base domain, not the `matrix.` domain, make sure you have permission from the room owner if that's not you).
Basic usage is like this: `!echo hi` or `!imgur puppies` or `!giphy matrix`
If you enabled the github_cmd service you can get the supported commands via `!github help`
You can also refer to the upstream [Documentation](https://github.com/matrix-org/go-neb).
If you would like Mjolnir to be able to deactivate users, move aliases, shutdown rooms, etc then it must be a server admin so you need to change `admin=no` to `admin=yes` in the command above.
## 2. Get an access token
If you use curl, you can get an access token like this:
```
curl -X POST --header 'Content-Type: application/json' -d '{
Alternatively, you can use a full-featured client (such as Element) to log in and get the access token from there (note: don't log out from the client as that will invalidate the token).
## 3. Make sure the account is free from rate limiting
You will need to prevent Synapse from rate limiting the bot's account. This is not an optional step. If you do not do this step Mjolnir will crash. [Currently there is no Synapse config option for this](https://github.com/matrix-org/synapse/issues/6286) so you have to manually edit the Synapse database. Manually editing the Synapse database is rarely a good idea but in this case it is required. Please ask for help if you are uncomfortable with these steps.
1. Copy the statement below into a text editor.
```
INSERT INTO ratelimit_override VALUES ('@bot.mjolnir:DOMAIN', 0, 0);
```
1. Change the username (`@bot.mjolnir:DOMAIN`) to the username you used when you registered the bot's account. You must change `DOMAIN` to your server's domain.
1. Get a database terminal by following these steps: [maintenance-postgres.md#getting-a-database-terminal](maintenance-postgres.md#getting-a-database-terminal)
1. Connect to Synapse's database by typing `\connect synapse` into the database terminal
1. Paste in the `INSERT INTO` command that you edited and press enter.
You can run `SELECT * FROM ratelimit_override;` to see if it worked. If the output looks like this:
Using your own account, create a new invite only room that you will use to manage the bot. This is the room where you will see the status of the bot and where you will send commands to the bot, such as the command to ban a user from another room. Anyone in this room can control the bot so it is important that you only invite trusted users to this room. The room must be unencrypted since the playbook does not support installing Pantalaimon yet.
Once you have created the room you need to copy the room ID so you can tell the bot to use that room. In Element you can do this by going to the room's settings, clicking Advanced, and then coping the internal room ID. The room ID will look something like `!QvgVuKq0ha8glOLGMG:DOMAIN`.
Finally invite the `@bot.mjolnir:DOMAIN` account you created earlier into the room.
## 5. Adjusting the playbook configuration
Add the following configuration to your `inventory/host_vars/matrix.DOMAIN/vars.yml` file (adapt to your needs):
You must replace `ACCESS_TOKEN_FROM_STEP_2_GOES_HERE` and `ROOM_ID_FROM_STEP_4_GOES_HERE` with the your own values.
You can refer to the upstream [documentation](https://github.com/matrix-org/mjolnir) for additional ways to use and configure mjolnir. Check out their [quickstart guide](https://github.com/matrix-org/mjolnir#quickstart-guide) for some basic commands you can give to the bot.
You can configure additional options by adding the `matrix_bot_mjolnir_configuration_extension_yaml` variable to your `inventory/host_vars/matrix.DOMAIN/vars.yml` file.
For example to change mjolnir's `recordIgnoredInvites` option to `true` you would add the following to your `vars.yml` file.
**Note**: bridging to [Discord](https://discordapp.com/) can also happen via the [mx-puppet-discord](configuring-playbook-bridge-mx-puppet-discord.md) bridge supported by the playbook.
The playbook can install and configure [matrix-appservice-discord](https://github.com/Half-Shot/matrix-appservice-discord) for you.
See the project's [documentation](https://github.com/Half-Shot/matrix-appservice-discord/blob/master/README.md) to learn what it does and why it might be useful to you.
## Setup Instructions
Instructions loosely based on [this](https://github.com/Half-Shot/matrix-appservice-discord#setting-up).
1. Create a Discord Application [here](https://discordapp.com/developers/applications).
2. Retrieve Client ID.
3. Create a bot from the Bot tab and retrieve the Bot token.
4. Enable the bridge with the following configuration in your `vars.yml` file:
5. If you've already installed Matrix services using the playbook before, you'll need to re-run it (`--tags=setup-all,start`). If not, proceed with [configuring other playbook services](configuring-playbook.md) and then with [Installing](installing.md). Get back to this guide once ready.
6. Retrieve Discord invite link from the `{{ matrix_appservice_discord_config_path }}/invite_link` file on the server (this defaults to `/matrix/appservice-discord/config/invite_link`). You need to peek at the file on the server via SSH, etc., because it's not available via HTTP(S).
7. Invite the Bot to Discord servers you wish to bridge. Administrator permission is recommended.
8. Room addresses follow this syntax: `#_discord_guildid_channelid`. You can easily find the guild and channel ids by logging into Discord in a browser and opening the desired channel. The URL will have this format: `discordapp.com/channels/guild_id/channel_id`. Once you have figured out the appropriate room addrss, you can join by doing `/join #_discord_guildid_channelid` in your Matrix client.
Other configuration options are available via the `matrix_appservice_discord_configuration_extension_yaml` variable.
## Getting Administrator access in a room
By default, you won't have Administrator access in rooms created by the bridge.
To [adjust room access privileges](#adjusting-room-access-privileges) or do various other things (change the room name subsequently, etc.), you'd wish to become an Administrator.
There's the Discord bridge's guide for [setting privileges on bridge managed rooms](https://github.com/Half-Shot/matrix-appservice-discord/blob/master/docs/howto.md#set-privileges-on-bridge-managed-rooms). To do the same with our container setup, run the following command on the server:
**Note**: bridging to [IRC](https://en.wikipedia.org/wiki/Internet_Relay_Chat) can also happen via the [Heisenbridge](configuring-playbook-bridge-heisenbridge.md) bridge supported by the playbook.
The playbook can install and configure the [matrix-appservice-irc](https://github.com/matrix-org/matrix-appservice-irc) bridge for you.
See the project's [documentation](https://github.com/matrix-org/matrix-appservice-irc/blob/master/HOWTO.md) to learn what it does and why it might be useful to you.
You'll need to use the following playbook configuration:
```yaml
matrix_appservice_irc_enabled: true
matrix_appservice_irc_ircService_servers:
irc.example.com:
name: "ExampleNet"
port: 6697
ssl: true
sasl: false
allowExpiredCerts: false
sendConnectionMessages: true
botConfig:
enabled: true
nick: "MatrixBot"
joinChannelsIfNoUsers: true
privateMessages:
enabled: true
federate: true
dynamicChannels:
enabled: true
createAlias: true
published: true
joinRule: public
groupId: +myircnetwork:localhost
federate: true
aliasTemplate: "#irc_$CHANNEL"
membershipLists:
enabled: false
floodDelayMs: 10000
global:
ircToMatrix:
initial: false
incremental: false
matrixToIrc:
initial: false
incremental: false
matrixClients:
userTemplate: "@irc_$NICK"
displayName: "$NICK (IRC)"
joinAttempts: -1
ircClients:
nickTemplate: "$DISPLAY[m]"
allowNickChanges: true
maxClients: 30
idleTimeout: 10800
reconnectIntervalMs: 5000
concurrentReconnectLimit: 50
lineLimit: 3
```
You then need to start a chat with `@irc_bot:YOUR_DOMAIN` (where `YOUR_DOMAIN` is your base domain, not the `matrix.` domain).
**Note**: bridging to [Slack](https://slack.com) can also happen via the [mx-puppet-slack](configuring-playbook-bridge-mx-puppet-slack.md) bridge supported by the playbook.
The playbook can install and configure [matrix-appservice-slack](https://github.com/matrix-org/matrix-appservice-slack) for you.
See the project's [documentation](https://github.com/matrix-org/matrix-appservice-slack/blob/master/README.md) to learn what it does and why it might be useful to you.
## Setup Instructions:
loosely based on [this](https://github.com/matrix-org/matrix-appservice-slack#Setup)
1. Create a new Matrix room to act as the administration control room. Note its internal room ID. This can
be done in Riot by making a message, opening the options for that message and choosing "view source". The
room ID will be displayed near the top.
2. Enable the bridge with the following configuration in your `vars.yml` file:
3. If you've already installed Matrix services using the playbook before, you'll need to re-run it (`--tags=setup-all,start`). If not, proceed with [configuring other playbook services](configuring-playbook.md) and then with [Installing](installing.md). Get back to this guide once ready.
4. Invite the bridge bot user into the admin room:
```
/invite @slackbot:MY.DOMAIN
```
Note that the bot's domain is your server's domain **without the `matrix.` prefix.**
5. Create a Classic Slack App [here](https://api.slack.com/apps?new_classic_app=1).
Name the app "matrixbot" (or anything else you'll remember).
Select the team/workspace this app will belong to.
Click on bot users and add a new bot user. We will use this account to bridge the the rooms.
6. Click on Event Subscriptions and enable them and use the request url `https://matrix.DOMAIN/appservice-slack`. Then add the following events and save:
Bot User Events:
- team_domain_change
- message.channels
- message.groups (if you want to bridge private channels)
- reaction_added
- reaction_removed
7. Click on OAuth & Permissions and add the following scopes:
- chat:write:bot
- users:read
- reactions:write
If you want to bridge files, also add the following:
- files:write:user
Note: In order to make Slack files visible to matrix users, this bridge will make Slack files visible to anyone with the url (including files in private channels). This is different than the current behavior in Slack, which only allows authenticated access to media posted in private channels. See MSC701 for details.
8. Click on Install App and Install App to Workspace. Note the access tokens shown. You will need the Bot User OAuth Access Token and if you want to bridge files, the OAuth Access Token whenever you link a room.
9. For each channel you would like to bridge, perform the following steps:
* Create a Matrix room in the usual manner for your client. Take a note of its Matrix room ID - it will look something like !aBcDeF:example.com.
* Invite the bot user to both the Slack and Matrix channels you would like to bridge using `/invite @matrixbot` for slack and `/invite @slackbot:MY.DOMAIN` for matrix.
* Determine the "channel ID" that Slack uses to identify the channel. You can see it when you open a given Slack channel in a browser. The URL reads like this: `https://app.slack.com/client/XXX/<the channel id>/details/`.
* Issue a link command in the administration control room with these collected values as arguments:
with file bridging:
```
link --channel_id CHANNELID --room !the-matrix:room.id --slack_bot_token xoxb-xxxxxxxxxx-xxxxxxxxxxxxxxxxxxxx --slack_user_token xoxp-xxxxxxxx-xxxxxxxxx-xxxxxxxx-xxxxxxxx
```
without file bridging:
```
link --channel_id CHANNELID --room !the-matrix:room.id --slack_bot_token xoxb-xxxxxxxxxx-xxxxxxxxxxxxxxxxxxxx
```
These arguments can be shortened to single-letter forms:
```
link -I CHANNELID -R !the-matrix:room.id -t xoxb-xxxxxxxxxx-xxxxxxxxxxxxxxxxxxxx
```
Other configuration options are available via the `matrix_appservice_slack_configuration_extension_yaml` variable.
10. Unlinking
Channels can be unlinked again like this:
```
unlink --room !the-matrix:room.id
```
Unlinking doesn't only disconnect the bridge, but also makes the slackbot leave the bridged matrix room. So in case you want to re-link later, don't forget to re-invite the slackbot into this room again.
## Troubleshooting
* as always, check the logs:
`journalctl -fu matrix-appservice-slack`
* linking: "Room is now pending-name"
This typically means that you haven't used the correct slack channel id. Unlink the room and recheck 'Determine the "channel ID"' from above.
* Messages work from M to S, but not the other way around
Check you logs, if they say something like
`WARN SlackEventHandler Ignoring message from unrecognised slack channel id : %s (%s) <the channel id> <some other id>`
then unlink your room, reinvite the bot and re-link it again. This may particularly hit you, if you tried to unsuccessfully link
your room multiple times without unlinking it after each failed attempt.
3. If you've already installed Matrix services using the playbook before, you'll need to re-run it (`--tags=setup-all,start`). If not, proceed with [configuring other playbook services](configuring-playbook.md) and then with [Installing](installing.md). Get back to this guide once ready.
4. If you're using the [Dimension Integration Manager](configuring-playbook-dimension.md), you can configure the Webhooks bridge by opening the Dimension integration manager -> Settings -> Bridges and selecting edit action for "Webhook Bridge". Press "Add self-hosted Bridge" button and populate "Provisioning URL" & "Shared Secret" values from `/matrix/appservice-webhooks/config/config.yaml` file's homeserver URL value and provisioning secret value, respectively.
5. Invite the bridge bot user to your room:
- either with `/invite @_webhook:<domain.name>` (*Note*: Make sure you have administration permissions in your room)
- or simply add the bridge bot to a private channel (personal channels imply you being an administrator)
6. Send a message to the bridge bot in order to receive a private message including the webhook link.
```
!webhook
```
7. The JSON body for posting messages will have to look like this:
The playbook can install and configure [beeper-linkedin](https://gitlab.com/beeper/linkedin) for you, for bridging to [LinkedIn](https://www.linkedin.com/) Messaging. This bridge is based on the mautrix-python framework and can be configured in a similar way to the other mautrix bridges
See the project's [documentation](https://gitlab.com/beeper/linkedin/-/blob/master/README.md) to learn what it does and why it might be useful to you.
```yaml
matrix_beeper_linkedin_enabled: true
```
There are some additional things you may wish to configure about the bridge before you continue.
Encryption support is off by default. If you would like to enable encryption, add the following to your `vars.yml` file:
You may wish to look at `roles/matrix-bridge-beeper-linkedin/templates/config.yaml.j2` to find other things you would like to configure.
## Set up Double Puppeting
If you'd like to use [Double Puppeting](https://docs.mau.fi/bridges/general/double-puppeting.html) (hint: you most likely do), you have 2 ways of going about it.
### Method 1: automatically, by enabling Shared Secret Auth
The bridge will automatically perform Double Puppeting if you enable [Shared Secret Auth](configuring-playbook-shared-secret-auth.md) for this playbook.
This is the recommended way of setting up Double Puppeting, as it's easier to accomplish, works for all your users automatically, and has less of a chance of breaking in the future.
## Usage
You then need to start a chat with `@linkedinbot:YOUR_DOMAIN` (where `YOUR_DOMAIN` is your base domain, not the `matrix.` domain).
Send `login YOUR_LINKEDIN_EMAIL_ADDRESS` to the bridge bot to enable bridging for your LinkedIn account.
If you run into trouble, check the [Troubleshooting](#troubleshooting) section below.
After successfully enabling bridging, you may wish to [set up Double Puppeting](#set-up-double-puppeting), if you haven't already done so.
## Troubleshooting
### Bridge asking for 2FA even if you don't have 2FA enabled
If you don't have 2FA enabled and are logging in from a strange IP for the first time, LinkedIn will send an email with a one-time code. You can use this code to authorize the bridge session. In my experience, once the IP is authorized, you will not be asked again.
**Note**: bridging to [IRC](https://en.wikipedia.org/wiki/Internet_Relay_Chat) can also happen via the [matrix-appservice-irc](configuring-playbook-bridge-appservice-irc.md) bridge supported by the playbook.
The playbook can install and configure [Heisenbridge](https://github.com/hifi/heisenbridge) - the bouncer-style [IRC](https://en.wikipedia.org/wiki/Internet_Relay_Chat) bridge for you.
See the project's [README](https://github.com/hifi/heisenbridge/blob/master/README.md) to learn what it does and why it might be useful to you. You can also take a look at [this demonstration video](https://www.youtube.com/watch?v=nQk1Bp4tk4I).
## Configuration
Below are the common configuration options that you may want to set, exhaustive list is in [the bridge's defaults var file](../roles/matrix-bridge-heisenbridge/defaults/main.yml).
At a minimum, you only need to enable the bridge to get it up and running (`inventory/host_vars/matrix.DOMAIN/vars.yml`):
```yaml
matrix_heisenbridge_enabled: true
# set owner (optional)
matrix_heisenbridge_owner: "@you:your-homeserver"
# to enable identd on host port 113/TCP (optional)
matrix_heisenbridge_identd_enabled: true
```
That's it! A registration file is automatically generated during the setup phase.
Setting the owner is optional as the first local user to DM `@heisenbridge:your-homeserver` will be made the owner.
If you are not using a local user you must set it as otherwise you can't DM it at all.
## Usage
After the bridge is successfully running just DM `@heisenbridge:your-homeserver` to start setting it up.
Help is available for all commands with the `-h` switch.
If the bridge ignores you and a DM is not accepted then the owner setting may be wrong.
You can also learn the basics by watching [this demonstration video](https://www.youtube.com/watch?v=nQk1Bp4tk4I).
If you encounter issues or feel lost you can join the project room at [#heisenbridge:vi.fi](https://matrix.to/#/#heisenbridge:vi.fi) for help.
You may wish to look at `roles/matrix-bridge-mautrix-facebook/templates/config.yaml.j2` to find other things you would like to configure.
## Set up Double Puppeting
If you'd like to use [Double Puppeting](https://docs.mau.fi/bridges/general/double-puppeting.html) (hint: you most likely do), you have 2 ways of going about it.
### Method 1: automatically, by enabling Shared Secret Auth
The bridge will automatically perform Double Puppeting if you enable [Shared Secret Auth](configuring-playbook-shared-secret-auth.md) for this playbook.
This is the recommended way of setting up Double Puppeting, as it's easier to accomplish, works for all your users automatically, and has less of a chance of breaking in the future.
### Method 2: manually, by asking each user to provide a working access token
**Note**: This method for enabling Double Puppeting can be configured only after you've already set up bridging (see [Usage](#usage)).
When using this method, **each user** that wishes to enable Double Puppeting needs to follow the following steps:
- retrieve a Matrix access token for yourself. You can use the following command:
- send the access token to the bot. Example: `login-matrix MATRIX_ACCESS_TOKEN_HERE`
- make sure you don't log out the `Mautrix-Facebook` device some time in the future, as that would break the Double Puppeting feature
## Usage
You then need to start a chat with `@facebookbot:YOUR_DOMAIN` (where `YOUR_DOMAIN` is your base domain, not the `matrix.` domain).
Send `login YOUR_FACEBOOK_EMAIL_ADDRESS` to the bridge bot to enable bridging for your Facebook Messenger account. You can learn more here about authentication from the bridge's [official documentation on Authentication](https://docs.mau.fi/bridges/python/facebook/authentication.html).
If you run into trouble, check the [Troubleshooting](#troubleshooting) section below.
After successfully enabling bridging, you may wish to [set up Double Puppeting](#set-up-double-puppeting), if you haven't already done so.
## Set up community-grouping
This is an **optional feature** that you may wish to enable.
The Facebook bridge can create a Matrix community for you, which would contain all your chats and contacts.
For this to work, the bridge's bot needs to have permissions to create communities (also referred to as groups).
Since the bot is a non-admin user, you need to enable such group-creation for non-privileged users in [Synapse's settings](configuring-playbook-synapse.md).
Once the bridge is restarted, it would create a community and invite you to it. You need to accept the community invitation manually.
If you don't see all your contacts, you may wish to send a `sync` message to the bot.
## Troubleshooting
### Facebook rejecting login attempts and forcing you to change password
If your Matrix server is in a wildly different location than where you usually use your Facebook account from, the bridge's login attempts may be outright rejected by Facebook. Along with that, Facebook may even force you to change the account's password.
If you happen to run into this problem while [setting up bridging](#usage), try to first get a successful session up by logging in to Facebook through the Matrix server's IP address.
The easiest way to do this may be to use [sshuttle](https://sshuttle.readthedocs.io/) to proxy your traffic through the Matrix server.
Example command for proxying your traffic through the Matrix server:
```
sshuttle -r root@matrix.DOMAIN:22 0/0
```
Once connected, you should be able to verify that you're browsing the web through the Matrix server's IP by checking [icanhazip](https://icanhazip.com/).
Then proceed to log in to [Facebook/Messenger](https://www.facebook.com/).
Once logged in, proceed to [set up bridging](#usage).
The playbook can install and configure [mautrix-googlechat](https://github.com/mautrix/googlechat) for you.
See the project's [documentation](https://docs.mau.fi/bridges/python/googlechat/index.html) to learn what it does and why it might be useful to you.
To enable the [Google Chat](https://chat.google.com/) bridge just use the following playbook configuration:
```yaml
matrix_mautrix_googlechat_enabled: true
```
## Set up Double Puppeting
If you'd like to use [Double Puppeting](https://docs.mau.fi/bridges/general/double-puppeting.html) (hint: you most likely do), you have 2 ways of going about it.
### Method 1: automatically, by enabling Shared Secret Auth
The bridge will automatically perform Double Puppeting if you enable [Shared Secret Auth](configuring-playbook-shared-secret-auth.md) for this playbook.
This is the recommended way of setting up Double Puppeting, as it's easier to accomplish, works for all your users automatically, and has less of a chance of breaking in the future.
### Method 2: manually, by asking each user to provide a working access token
**Note**: This method for enabling Double Puppeting can be configured only after you've already set up bridging (see [Usage](#usage)).
When using this method, **each user** that wishes to enable Double Puppeting needs to follow the following steps:
- retrieve a Matrix access token for yourself. You can use the following command:
- send the access token to the bot. Example: `login-matrix MATRIX_ACCESS_TOKEN_HERE`
- make sure you don't log out the `Mautrix-googlechat` device some time in the future, as that would break the Double Puppeting feature
## Usage
Once the bot is enabled you need to start a chat with `googlechat bridge bot` with handle `@googlechatbot:YOUR_DOMAIN` (where `YOUR_DOMAIN` is your base domain, not the `matrix.` domain).
Send `login` to the bridge bot to receive a link to the portal from which you can enable the bridging. Open the link sent by the bot and follow the instructions.
Automatic login may not work. If it does not, reload the page and select the "Manual login" checkbox before starting. Manual login involves logging into your Google account normally and then manually getting the OAuth token from browser cookies with developer tools.
Once logged in, recent chats should show up as new conversations automatically. Other chats will get portals as you receive messages.
You can learn more about authentication from the bridge's [official documentation on Authentication](https://docs.mau.fi/bridges/python/googlechat/authentication.html).
After successfully enabling bridging, you may wish to [set up Double Puppeting](#set-up-double-puppeting), if you haven't already done so.
# The [Mautrix Hangouts Bridge](https://mau.dev/mautrix/hangouts) is no longer maintained. It has changed to a [Google Chat Bridge](https://github.com/mautrix/googlechat). Setup instructions for the Google Chat Bridge can be [found here](configuring-playbook-bridge-mautrix-googlechat.md).
# Setting up Mautrix Hangouts (optional)
The playbook can install and configure [mautrix-hangouts](https://github.com/mautrix/hangouts) for you.
See the project's [documentation](https://docs.mau.fi/bridges/python/hangouts/index.html) to learn what it does and why it might be useful to you.
To enable the [Google Hangouts](https://hangouts.google.com/) bridge just use the following playbook configuration:
```yaml
matrix_mautrix_hangouts_enabled: true
```
## Set up Double Puppeting
If you'd like to use [Double Puppeting](https://docs.mau.fi/bridges/general/double-puppeting.html) (hint: you most likely do), you have 2 ways of going about it.
### Method 1: automatically, by enabling Shared Secret Auth
The bridge will automatically perform Double Puppeting if you enable [Shared Secret Auth](configuring-playbook-shared-secret-auth.md) for this playbook.
This is the recommended way of setting up Double Puppeting, as it's easier to accomplish, works for all your users automatically, and has less of a chance of breaking in the future.
### Method 2: manually, by asking each user to provide a working access token
**Note**: This method for enabling Double Puppeting can be configured only after you've already set up bridging (see [Usage](#usage)).
When using this method, **each user** that wishes to enable Double Puppeting needs to follow the following steps:
- retrieve a Matrix access token for yourself. You can use the following command:
- send the access token to the bot. Example: `login-matrix MATRIX_ACCESS_TOKEN_HERE`
- make sure you don't log out the `Mautrix-Hangouts` device some time in the future, as that would break the Double Puppeting feature
## Usage
Once the bot is enabled you need to start a chat with `Hangouts bridge bot` with handle `@hangoutsbot:YOUR_DOMAIN` (where `YOUR_DOMAIN` is your base domain, not the `matrix.` domain).
Send `login` to the bridge bot to receive a link to the portal from which you can enable the bridging. Open the link sent by the bot and follow the instructions.
Automatic login may not work. If it does not, reload the page and select the "Manual login" checkbox before starting. Manual login involves logging into your Google account normally and then manually getting the OAuth token from browser cookies with developer tools.
Once logged in, recent chats should show up as new conversations automatically. Other chats will get portals as you receive messages.
You can learn more about authentication from the bridge's [official documentation on Authentication](https://docs.mau.fi/bridges/python/hangouts/authentication.html).
After successfully enabling bridging, you may wish to [set up Double Puppeting](#set-up-double-puppeting), if you haven't already done so.
The playbook can install and configure [mautrix-instagram](https://github.com/mautrix/instagram) for you.
See the project's [documentation](https://docs.mau.fi/bridges/python/instagram/index.html) to learn what it does and why it might be useful to you.
```yaml
matrix_mautrix_instagram_enabled: true
```
## Usage
You then need to start a chat with `@instagrambot:YOUR_DOMAIN` (where `YOUR_DOMAIN` is your base domain, not the `matrix.` domain).
Send `login YOUR_INSTAGRAM_EMAIL_ADDRESS YOUR_INSTAGRAM_PASSWORD` to the bridge bot to enable bridging for your instagram/Messenger account.
You can learn more here about authentication from the bridge's [official documentation on Authentication](https://docs.mau.fi/bridges/python/instagram/authentication.html).
The playbook can install and configure [mautrix-signal](https://github.com/mautrix/signal) for you.
See the project's [documentation](https://docs.mau.fi/bridges/python/signal/index.html) to learn what it does and why it might be useful to you.
**Note/Prerequisite**: If you're running with the Postgres database server integrated by the playbook (which is the default), you don't need to do anything special and can easily proceed with installing. However, if you're [using an external Postgres server](configuring-playbook-external-postgres.md), you'd need to manually prepare a Postgres database for this bridge and adjust the variables related to that (`matrix_mautrix_signal_database_*`).
Use the following playbook configuration:
```yaml
matrix_mautrix_signal_enabled: true
```
There are some additional things you may wish to configure about the bridge before you continue.
The relay bot functionality is off by default. If you would like to enable the relay bot, add the following to your `vars.yml` file:
```yaml
matrix_mautrix_signal_relaybot_enabled: true
```
If you want to activate the relay bot in a room, use `!signal set-relay`.
Use `!signal unset-relay` to deactivate.
By default, any user on your homeserver will be able to use the bridge.
If you enable the relay bot functionality, it will relay every user's messages in a portal room - no matter which homeserver they're from.
Different levels of permission can be granted to users:
* relay - Allowed to be relayed through the bridge, no access to commands;
* user - Use the bridge with puppeting;
* admin - Use and administer the bridge.
The permissions are following the sequence: nothing <relay<user<admin.
The default permissions are set as follows:
```yaml
permissions:
'*': relay
YOUR_DOMAIN: user
```
If you want to augment the preset permissions, you might want to set the additional permissions with the following settings in your `vars.yml` file:
This will add the admin permission to the specific user, while keepting the default permissions.
In case you want to replace the default permissions settings **completely**, populate the following item within your `vars.yml` file:
```yaml
matrix_mautrix_signal_bridge_permissions: |
'@ADMIN:YOUR_DOMAIN': admin
'@USER:YOUR_DOMAIN' : user
```
You may wish to look at `roles/matrix-bridge-mautrix-signal/templates/config.yaml.j2` to find more information on the permissions settings and other options you would like to configure.
## Set up Double Puppeting
If you'd like to use [Double Puppeting](https://docs.mau.fi/bridges/general/double-puppeting.html) (hint: you most likely do), you have 2 ways of going about it.
### Method 1: automatically, by enabling Shared Secret Auth
The bridge will automatically perform Double Puppeting if you enable [Shared Secret Auth](configuring-playbook-shared-secret-auth.md) for this playbook.
This is the recommended way of setting up Double Puppeting, as it's easier to accomplish, works for all your users automatically, and has less of a chance of breaking in the future.
### Method 2: manually, by asking each user to provide a working access token
**Note**: This method for enabling Double Puppeting can be configured only after you've already set up bridging (see [Usage](#usage)).
When using this method, **each user** that wishes to enable Double Puppeting needs to follow the following steps:
- retrieve a Matrix access token for yourself. You can use the following command:
If you'd like to use [Double Puppeting](https://docs.mau.fi/bridges/general/double-puppeting.html) (hint: you most likely do), you have 2 ways of going about it.
### Method 1: automatically, by enabling Shared Secret Auth
The bridge will automatically perform Double Puppeting if you enable [Shared Secret Auth](configuring-playbook-shared-secret-auth.md) for this playbook.
This is the recommended way of setting up Double Puppeting, as it's easier to accomplish, works for all your users automatically, and has less of a chance of breaking in the future.
### Method 2: manually, by asking each user to provide a working access token
**Note**: This method for enabling Double Puppeting can be configured only after you've already set up bridging.
When using this method, **each user** that wishes to enable Double Puppeting needs to follow the following steps:
- retrieve a Matrix access token for yourself. You can use the following command:
- send `login-matrix` to the bot and follow instructions about how to send the access token to it
- make sure you don't log out the `Mautrix-Telegram` device some time in the future, as that would break the Double Puppeting feature
## Usage
You then need to start a chat with `@telegrambot:YOUR_DOMAIN` (where `YOUR_DOMAIN` is your base domain, not the `matrix.` domain).
If you want to use the relay-bot feature ([relay bot documentation](https://docs.mau.fi/bridges/python/telegram/relay-bot.html)), which allows anonymous user to chat with telegram users, use the following additional playbook configuration:
The playbook can install and configure [mautrix-whatsapp](https://github.com/mautrix/whatsapp) for you.
See the project's [documentation](https://docs.mau.fi/bridges/go/whatsapp/index.html) to learn what it does and why it might be useful to you.
Use the following playbook configuration:
```yaml
matrix_mautrix_whatsapp_enabled: true
```
## Set up Double Puppeting
If you'd like to use [Double Puppeting](https://docs.mau.fi/bridges/general/double-puppeting.html) (hint: you most likely do), you have 2 ways of going about it.
### Method 1: automatically, by enabling Shared Secret Auth
The bridge will automatically perform Double Puppeting if you enable [Shared Secret Auth](configuring-playbook-shared-secret-auth.md) for this playbook.
This is the recommended way of setting up Double Puppeting, as it's easier to accomplish, works for all your users automatically, and has less of a chance of breaking in the future.
### Method 2: manually, by asking each user to provide a working access token
**Note**: This method for enabling Double Puppeting can be configured only after you've already set up bridging (see [Usage](#usage)).
When using this method, **each user** that wishes to enable Double Puppeting needs to follow the following steps:
- retrieve a Matrix access token for yourself. You can use the following command:
**Note**: bridging to [Discord](https://discordapp.com/) can also happen via the [matrix-appservice-discord](configuring-playbook-bridge-appservice-discord.md) bridge supported by the playbook.
The playbook can install and configure
[mx-puppet-discord](https://github.com/matrix-discord/mx-puppet-discord) for you.
See the project page to learn what it does and why it might be useful to you.
To enable the [Discord](https://discordapp.com/) bridge just use the following
playbook configuration:
```yaml
matrix_mx_puppet_discord_enabled: true
```
## Usage
Once the bot is enabled you need to start a chat with `Discord Puppet Bridge` with
the handle `@_discordpuppet_bot:YOUR_DOMAIN` (where `YOUR_DOMAIN` is your base
domain, not the `matrix.` domain).
Three authentication methods are available, Legacy Token, OAuth and xoxc token.
See mx-puppet-discord [documentation](https://github.com/matrix-discord/mx-puppet-discord)
for more information about how to configure the bridge.
Once logged in, send `list` to the bot user to list the available rooms.
Clicking rooms in the list will result in you receiving an invitation to the
bridged room.
Also send `help` to the bot to see the commands available.
[mx-puppet-groupme](https://gitlab.com/robintown/mx-puppet-groupme) for you.
See the project page to learn what it does and why it might be useful to you.
To enable the [GroupMe](https://groupme.com/) bridge just use the following
playbook configuration:
```yaml
matrix_mx_puppet_groupme_enabled: true
```
## Usage
Once the bot is enabled you need to start a chat with `GroupMe Puppet Bridge` with
the handle `@_groupmepuppet_bot:YOUR_DOMAIN` (where `YOUR_DOMAIN` is your base
domain, not the `matrix.` domain).
One authentication method is available.
To link your GroupMe account, go to [dev.groupme.com](https://dev.groupme.com/), sign in, and select "Access Token" from the top menu. Copy the token and message the bridge with:
```
link <accesstoken>
```
Once logged in, send `listrooms` to the bot user to list the available rooms.
Clicking rooms in the list will result in you receiving an invitation to the
bridged room.
Also send `help` to the bot to see the commands available.
**Note**: bridging to [Slack](https://slack.com) can also happen via the [matrix-appservice-slack](configuring-playbook-bridge-appservice-slack.md) bridge supported by the playbook.
The playbook can install and configure
[mx-puppet-slack](https://github.com/Sorunome/mx-puppet-slack) for you.
See the project page to learn what it does and why it might be useful to you.
To enable the [Slack](https://slack.com/) bridge just use the following
playbook configuration:
```yaml
matrix_mx_puppet_slack_enabled: true
```
## Usage
Once the bot is enabled you need to start a chat with `Slack Puppet Bridge` with
the handle `@_slackpuppet_bot:YOUR_DOMAIN` (where `YOUR_DOMAIN` is your base
domain, not the `matrix.` domain).
Three authentication methods are available, Legacy Token, OAuth and xoxc token.
See mx-puppet-slack [documentation](https://github.com/Sorunome/mx-puppet-slack)
for more information about how to configure the bridge.
Once logged in, send `list` to the bot user to list the available rooms.
Clicking rooms in the list will result in you receiving an invitation to the
bridged room.
Also send `help` to the bot to see the commands available.
By default, this playbook installs the [Element](https://github.com/vector-im/element-web) Matrix client web application.
If that's okay, you can skip this document.
## Disabling Element
If you'd like for the playbook to not install Element (or to uninstall it if it was previously installed), you can disable it in your configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`):
```yaml
matrix_client_element_enabled: false
```
## Configuring Element settings
The playbook provides some customization variables you could use to change Element's settings.
Their defaults are defined in [`roles/matrix-client-element/defaults/main.yml`](../roles/matrix-client-element/defaults/main.yml) and they ultimately end up in the generated `/matrix/element/config.json` file (on the server). This file is generated from the [`roles/matrix-client-element/templates/config.json.j2`](../roles/matrix-client-element/templates/config.json.j2) template.
**If there's an existing variable** which controls a setting you wish to change, you can simply define that variable in your configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`) and [re-run the playbook](installing.md) to apply the changes.
Alternatively, **if there is no pre-defined variable** for an Element setting you wish to change:
- you can either **request a variable to be created** (or you can submit such a contribution yourself). Keep in mind that it's **probably not a good idea** to create variables for each one of Element's various settings that rarely get used.
- or, you can **extend and override the default configuration** ([`config.json.j2`](../roles/matrix-client-element/templates/config.json.j2)) by making use of the `matrix_client_element_configuration_extension_json_` variable. You can find information about this in [`roles/matrix-client-element/defaults/main.yml`](../roles/matrix-client-element/defaults/main.yml).
- or, if extending the configuration is still not powerful enough for your needs, you can **override the configuration completely** using `matrix_client_element_configuration_default` (or `matrix_client_element_configuration`). You can find information about this in [`roles/matrix-client-element/defaults/main.yml`](../roles/matrix-client-element/defaults/main.yml).
## Themes
To change the look of Element, you can define your own themes manually by using the `matrix_client_element__settingDefaults_custom_themes` setting.
Or better yet, you can automatically pull it all themes provided by the [aaronraimist/element-themes](https://github.com/aaronraimist/element-themes) project by simply flipping a flag (`matrix_client_element_themes_enabled: true`).
If you make your own theme, we encourage you to submit it to the **aaronraimist/element-themes** project, so that the whole community could easily enjoy it.
Note that for a custom theme to work well, all Element instances that you use must have the same theme installed.
**[Dimension](https://dimension.t2bot.io) can only be installed after Matrix services are installed and running.**
If you're just installing Matrix services for the first time, please continue with the [Configuration](configuring-playbook.md) / [Installation](installing.md) flow and come back here later.
**Note**: This playbook now supports running [Dimension](https://dimension.t2bot.io) in both a federated and [unfederated](https://github.com/turt2live/matrix-dimension/blob/master/docs/unfederated.md) environments. This is handled automatically based on the value of `matrix_synapse_federation_enabled`. Enabling Dimension, means that the `openid` API endpoints will be exposed on the Matrix Federation port (usually `8448`), even if [federation](configuring-playbook-federation.md) is disabled. It's something to be aware of, especially in terms of firewall whitelisting (make sure port `8448` is accessible).
## Prerequisites
The `dimension.<your-domain>` DNS record must be created. See [Configuring your DNS server](configuring-dns.md) on how to set up DNS record correctly.
## Enable
[Dimension integrations manager](https://dimension.t2bot.io) installation is disabled by default. You can enable it in your configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`):
```yaml
matrix_dimension_enabled: true
```
## Define admin users
These users can modify the integrations this Dimension supports. Admin interface is accessible at `https://dimension.<your-domain>/riot-app/admin` after logging in to element.
Add this to your configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`):
```yaml
matrix_dimension_admins:
- "@user1:{{ matrix_domain }}"
- "@user2:{{ matrix_domain }}"
```
## Access token
We recommend that you create a dedicated Matrix user for Dimension (`dimension` is a good username).
Follow our [Registering users](registering-users.md) guide to learn how to register **a regular (non-admin) user**.
You are required to specify an access token (belonging to this new user) for Dimension to work.
To get an access token for the Dimension user, you can follow one of two options:
*Through an interactive login*:
1. In a private browsing session (incognito window), open Element.
1. Log in with the `dimension` user and its password.
1. Set the display name and avatar, if required.
1. In the settings page choose "Help & About", scroll down to the bottom and expand the `Access Token` section.
1. Copy the access token to your configuration.
1. Close the private browsing session. **Do not log out**. Logging out will invalidate the token, making it not work.
*With CURL*
```
curl -X POST --header 'Content-Type: application/json' -d '{
After Dimension has been installed you may need to log out and log back in for it to pick up the new integrations manager. Then you can access integrations in Element by opening a room, clicking the Room info button (`i`) button in the top right corner of the screen, and then clicking Add widgets, bridges & bots.
## Jitsi domain
By default Dimension will use [jitsi.riot.im](https://jitsi.riot.im/) as the `conferenceDomain` of [Jitsi](https://jitsi.org/) audio/video conference widgets. For users running [a self-hosted Jitsi instance](./configuring-playbook-jitsi.md), you will likely want the widget to use your own Jitsi instance. Currently there is no way to configure this via the playbook, see [this issue](https://github.com/turt2live/matrix-dimension/issues/345) for details.
In the interim until the above limitation is resolved, an admin user needs to configure the domain via the admin ui once dimension is running. In Element, go to *Manage Integrations*→*Settings*→*Widgets*→*Jitsi Conference Settings* and set *Jitsi Domain* and *Jitsi Script URL* appropriately.
## Additional features
To use a more custom configuration, you can define a `matrix_dimension_configuration_extension_yaml` string variable and put your configuration in it.
To learn more about how to do this, refer to the information about `matrix_dimension_configuration_extension_yaml` in the [default variables file](../roles/matrix-dimension/defaults/main.yml) of the Dimension component.
You can find all configuration options on [GitHub page of Dimension project](https://github.com/turt2live/matrix-dimension/blob/master/config/default.yaml).
By default, this playbook sets up an [Exim](https://www.exim.org/) email server through which all Matrix services send emails.
The email server would attempt to deliver emails directly to their final destination.
This may or may not work, depending on your domain configuration (SPF settings, etc.)
By default, emails are sent from `matrix@<your-domain-name>` (as specified by the `matrix_mailer_sender_address` playbook variable).
**Note**: If you are using a Google Cloud instance, [port 25 is always blocked](https://cloud.google.com/compute/docs/tutorials/sending-mail/), so you need to relay email through another SMTP server as described below.
## Firewall settings
No matter whether you send email directly (the default) or you relay email through another host (see how below), you'll probably need to allow outgoing traffic for TCP ports 25/587 (depending on configuration).
## Relaying email through another SMTP server
If you'd like to relay email through another SMTP server, feel free to redefine a few playbook variables.
**Note**: only the secure submission protocol (using `STARTTLS`, usually on port `587`) is supported. **SMTPS** (encrypted SMTP, usually on port `465`) **is not supported**.
### Configuations for sending emails using Sendgrid
An easy and free SMTP service to set up is [Sendgrid](https://sendgrid.com/), the free tier allows for up to 100 emails per day to be sent. In the settings below you can provide any email for `matrix_mailer_sender_address`.
The only other thing you need to change is the `matrix_mailer_relay_auth_password`, which you can generate at https://app.sendgrid.com/settings/api_keys. The API key password looks something like `SG.955oW1mLSfwds7i9Yd6IA5Q.q8GTaB8q9kGDzasegdG6u95fQ-6zkdwrPP8bOeuI`.
Note that the `matrix_mailer_relay_auth_username` is literally the string `apikey`, it's always the same for Sendgrid.
The playbook can install and configure [email2matrix](https://github.com/devture/email2matrix) for you.
See the project's [documentation](https://github.com/devture/email2matrix/blob/master/docs/README.md) to learn what it does and why it might be useful to you.
## Preparation
### Port availability
Ensure that port 25 is available on your Matrix server and open in your firewall.
If you have `postfix` or some other email server software installed, you may need to manually remove it first (unless you need it, of course).
If you really need to run an email server on the Matrix machine for other purposes, it may be possible to run Email2Matrix on another port (with a configuration like `matrix_email2matrix_smtp_host_bind_port: "127.0.0.01:2525"`) and have your other email server relay messages there.
For details about using Email2Matrix alongside [Postfix](http://www.postfix.org/), see [here](https://github.com/devture/email2matrix/blob/master/docs/setup_with_postfix.md).
### Creating a user
Before enabling Email2Matrix, you'd most likely wish to create a dedicated user (or more) that would be sending messages on the Matrix side.
Refer to [Registering users](registering-users.md) for ways to do that. A regular (non-admin) user works best.
### Creating a shared room
After creating a sender user, you should create one or more Matrix rooms that you share with that user.
It doesn't matter who creates and owns the rooms and who joins later (you or the sender user).
What matters is that both you and the sender user are part of the same room and that the sender user has enough privileges in the room to be able to send messages there.
Inviting additional people to the room is okay too.
Take note of each room's room id (different clients show the room id in a different place).
You'll need the room id when doing [Configuration](#configuration) below.
### Obtaining an access token for the sender user
In order for the sender user created above to be able to send messages to the room, we'll need to obtain an access token for it.
[Etherpad](https://etherpad.org) is is an open source collaborative text editor that can be embedded in a Matrix chat room using the [Dimension integrations manager](https://dimension.t2bot.io)
When enabled together with the Jitsi audio/video conferencing system (see [our docs on Jitsi](configuring-playbook-jitsi.md)), it will be made available as an option during the conferences.
## Prerequisites
For the self-hosted Etherpad instance to be available to your users, you must first enable and configure the **Dimension integrations manager** as described in [the playbook documentation](configuring-playbook-dimension.md)
## Installing
[Etherpad](https://etherpad.org) installation is disabled by default. You can enable it in your configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`):
```yaml
matrix_etherpad_enabled: true
```
## Set Dimension default to the self-hosted Etherpad
The Dimension administrator users can configure the default URL template. The Dimension configuration menu can be accessed with the sprocket icon as you begin to add a widget to a room in Element. There you will find the Etherpad Widget Configuration action beneath the _Widgets_ tab. Replace `scalar.vector.im` with your own Dimension domain.
### Removing the integrated Etherpad chat
If you wish to disable the Etherpad chat button, you can do it by appending `?showChat=false` to the end of the pad URL, or the template.
If your Etherpad widget fails to load, this might be due to Dimension generating a Pad name so long, the Etherpad app rejects it.
`$roomId_$padName` can end up being longer than 50 characters. You can avoid having this problem by altering the template so it only contains the three word random identifier `$padName`.
By default, this playbook would set up a PostgreSQL database server on your machine, running in a Docker container.
If that's alright, you can skip this.
If you'd like to use an external PostgreSQL server that you manage, you can edit your configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`).
It should be something like this:
```yaml
matrix_postgres_enabled: false
# Rewire Synapse to use your external Postgres server
The database (as specified in `matrix_synapse_database_database`) must exist and be accessible with the given credentials.
It must be empty or contain a valid Synapse database. If empty, Synapse would populate it the first time it runs.
**Note**: the external server that you specify in `matrix_synapse_database_host` must be accessible from within the `matrix-synapse` Docker container (and possibly other containers too). This means that it either needs to be a publicly accessible hostname or that it's a hostname on the same Docker network where all containers installed by this playbook run (a network called `matrix` by default). Using a local PostgreSQL instance on the host (running on the same machine, but not in a container) is not possible.
The connection to your external Postgres server **will not be SSL encrypted**, as [we don't support that yet](https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/89).
By default, your server federates with the whole Matrix network.
That is, people on your server can communicate with people on any other Matrix server.
## Federating only with select servers
To make your server only federate with servers of your choosing, add this to your configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`):
```yaml
matrix_synapse_federation_domain_whitelist:
- example.com
- another.com
```
If you wish to disable federation, you can do that with an empty list (`[]`), or better yet by completely disabling federation (see below).
## Exposing the room directory over federation
By default, your server's public rooms directory is not exposed to other servers via federation.
If you wish to expose it, add this to your configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`):
To completely disable federation, isolating your server from the rest of the Matrix network, add this to your configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`):
```yaml
matrix_synapse_federation_enabled: false
```
With that, your server's users will only be able to talk among themselves, but not to anyone who is on another server.
**Disabling federation does not necessarily disable the federation port** (`8448`). Services like [Dimension](configuring-playbook-dimension.md) and [ma1sd](configuring-playbook-ma1sd.md) normally rely on `openid` APIs exposed on that port. Even if you disable federation and only if necessary, we may still be exposing the federation port and serving the `openid` APIs there. To override this and completely disable Synapse's federation port use:
```yaml
# This stops the federation port on the Synapse side (normally `matrix-synapse:8048` on the container network).
matrix_synapse_federation_port_enabled: false
# This removes the `8448` virtual host from the matrix-nginx-proxy reverse-proxy server.
The playbook can install the [Jitsi](https://jitsi.org/) video-conferencing platform and integrate it with [Element](configuring-playbook-client-element.md).
Jitsi installation is **not enabled by default**, because it's not a core component of Matrix services.
The setup done by the playbook is very similar to [docker-jitsi-meet](https://github.com/jitsi/docker-jitsi-meet). You can refer to the documentation there for many of the options here.
## Prerequisites
Before installing Jitsi, make sure you've created the `jitsi.DOMAIN` DNS record. See [Configuring DNS](configuring-dns.md).
You may also need to open the following ports to your server:
- `4443/tcp` - RTP media fallback over TCP
- `10000/udp` - RTP media over UDP. Depending on your firewall/NAT setup, incoming RTP packets on port `10000` may have the external IP of your firewall as destination address, due to the usage of STUN in JVB (see [`matrix_jitsi_jvb_stun_servers`](../roles/matrix-jitsi/defaults/main.yml)).
## Installation
Add this to your `inventory/host_vars/matrix.DOMAIN/vars.yml` configuration:
```yaml
matrix_jitsi_enabled: true
# Run `bash inventory/scripts/jitsi-generate-passwords.sh` to generate these passwords,
# or define your own strong passwords manually.
matrix_jitsi_jicofo_auth_password: ""
matrix_jitsi_jvb_auth_password: ""
matrix_jitsi_jibri_recorder_password: ""
matrix_jitsi_jibri_xmpp_password: ""
```
## (Optional) Configure Jitsi authentication and guests mode
By default the Jitsi Meet instance does not require any kind of login and is open to use for anyone without registration.
If you're fine with such an open Jitsi instance, please skip to [Apply changes](#apply-changes).
If you would like to control who is allowed to open meetings on your new Jitsi instance, then please follow this step to enable Jitsi's authentication and guests mode. With authentication enabled, all meeting rooms have to be opened by a registered user, after which guests are free to join. If a registered host is not yet present, guests are put on hold in individual waiting rooms.
Add these two lines to your `inventory/host_vars/matrix.DOMAIN/vars.yml` configuration:
```yaml
matrix_jitsi_enable_auth: true
matrix_jitsi_enable_guests: true
```
### (Optional) LDAP authentication
The default authentication mode of Jitsi is `internal`, however LDAP is also supported. An example LDAP configuration could be:
For more information refer to the [docker-jitsi-meet](https://github.com/jitsi/docker-jitsi-meet#authentication-using-ldap) and the [saslauthd `LDAP_SASLAUTHD`](https://github.com/winlibs/cyrus-sasl/blob/master/saslauthd/LDAP_SASLAUTHD) documentation.
## (Optional) Making your Jitsi server work on a LAN
By default the Jitsi Meet instance does not work with a client in LAN (Local Area Network), even if others are connected from WAN. There are no video and audio. In the case of WAN to WAN everything is ok.
The reason is the Jitsi VideoBridge git to LAN client the IP address of the docker image instead of the host. The [documentation](https://github.com/jitsi/docker-jitsi-meet#running-behind-nat-or-on-a-lan-environment) of Jitsi in docker suggest to add `DOCKER_HOST_ADDRESS` in enviornment variable to make it work.
Here is how to do it in the playbook.
Add these two lines to your `inventory/host_vars/matrix.DOMAIN/vars.yml` configuration:
You may want to **suspend unused video layers** until they are requested again, to save up resources on both server and clients.
Read more on this feature [here](https://jitsi.org/blog/new-off-stage-layer-suppression-feature/)
For this add this line to your `inventory/host_vars/matrix.DOMAIN/vars.yml` configuration:
You may wish to **disable audio levels** to avoid excessive refresh of the client-side page and decrease the CPU consumption involved.
You may want to **limit the number of video feeds forwarded to each client**, to save up resources on both server and clients. As clients’ bandwidth and CPU may not bear the load, use this setting to avoid lag and crashes.
This feature is found by default in other webconference applications such as Office 365 Teams (limit is set to 4).
Read how it works [here](https://github.com/jitsi/jitsi-videobridge/blob/master/doc/last-n.md) and performance evaluation on this [study](https://jitsi.org/wp-content/uploads/2016/12/nossdav2015lastn.pdf).
You may want to **limit the maximum video resolution**, to save up resources on both server and clients.
## Apply changes
Then re-run the playbook: `ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,start`
## Required if configuring Jitsi with internal authentication: register new users
Until this gets integrated into the playbook, we need to register new users / meeting hosts for Jitsi manually.
Please SSH into your matrix host machine and execute the following command targeting the `matrix-jitsi-prosody` container:
Run this command for each user you would like to create, replacing `<USERNAME>` and `<PASSWORD>` accordingly. After you've finished, please exit the host.
**If you get an error** like this: "Error: Account creation/modification not supported.", it's likely that you had previously installed Jitsi without auth/guest support. In such a case, you should look into [Rebuilding your Jitsi installation](#rebuilding-your-jitsi-installation).
## Usage
You can use the self-hosted Jitsi server in multiple ways:
- **by adding a widget to a room via Element** (the one configured by the playbook at `https://element.DOMAIN`). Just start a voice or a video call in a room containing more than 2 members and that would create a Jitsi widget which utilizes your self-hosted Jitsi server.
- **by adding a widget to a room via the Dimension Integration Manager**. You'll have to point the widget to your own Jitsi server manually. See our [Dimension](./configuring-playbook-dimension.md) documentation page for more details. Naturally, Dimension would need to be installed first (the playbook doesn't install it by default).
- **directly (without any Matrix integration)**. Just go to `https://jitsi.DOMAIN`
**Note**: Element apps on mobile devices currently [don't support joining meetings on a self-hosted Jitsi server](https://github.com/vector-im/riot-web/blob/601816862f7d84ac47547891bd53effa73d32957/docs/jitsi.md#mobile-app-support).
## Troubleshooting
### Rebuilding your Jitsi installation
**If you ever run into any trouble** or **if you change configuration (`matrix_jitsi_*` variables) too much**, we urge you to rebuild your Jitsi setup.
We normally don't require such manual intervention for other services, but Jitsi services generate a lot of configuration files on their own.
These files are not all managed by Ansible (at least not yet), so you may sometimes need to delete them all and start fresh.
To rebuild your Jitsi configuration:
- SSH into the server and do this:
- stop all Jitsi services (`systemctl stop matrix-jitsi-*`).
- remove all Jitsi configuration & data (`rm -rf /matrix/jitsi`)
- ask Ansible to set up Jitsi anew and restart services (`ansible-playbook -i inventory/hosts setup.yml --tags=setup-jitsi,start`)
# Setting up the LDAP authentication password provider module (optional, advanced)
The playbook can install and configure the [matrix-synapse-ldap3](https://github.com/matrix-org/matrix-synapse-ldap3) LDAP Auth password provider for you.
See that project's documentation to learn what it does and why it might be useful to you.
If you decide that you'd like to let this playbook install it for you, you need some configuration like this:
If you wish for users to **authenticate only against configured password providers** (like this one), **without consulting Synapse's local database**, feel free to disable it:
If you wish to use the ma1sd Identity Server for LDAP authentication instead of [matrix-synapse-ldap3](https://github.com/matrix-org/matrix-synapse-ldap3) consult [Adjusting ma1sd Identity Server configuration](configuring-playbook-ma1sd.md#authentication).
# Adjusting ma1sd Identity Server configuration (optional)
By default, this playbook configures an [ma1sd](https://github.com/ma1uta/ma1sd) Identity Server for you.
This server is private by default, potentially at the expense of user discoverability.
*ma1sd is a fork of [mxisd](https://github.com/kamax-io/mxisd) which was pronounced end of life 2019-06-21.*
**Note**: enabling ma1sd (which is also the default), means that the `openid` API endpoints will be exposed on the Matrix Federation port (usually `8448`), even if [federation](configuring-playbook-federation.md) is disabled. It's something to be aware of, especially in terms of firewall whitelisting (make sure port `8448` is accessible).
## Disabling ma1sd
ma1sd, being an Identity Server, is not strictly needed. It is only used for 3PIDs (3rd party identifiers like E-mail and phone numbers) and some [enhanced features](https://github.com/ma1uta/ma1sd/#features).
If you'd like for the playbook to not install ma1sd (or to uninstall it if it was previously installed), you can disable it in your configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`):
```yaml
matrix_ma1sd_enabled: false
```
## Matrix.org lookup forwarding
To ensure maximum discovery, you can make your identity server also forward lookups to the central matrix.org Identity server (at the cost of potentially leaking all your contacts information).
Enabling this is discouraged and you'd better [learn more](https://github.com/ma1uta/ma1sd/blob/master/docs/features/identity.md#lookups) before proceeding.
Enabling matrix.org forwarding can happen with the following configuration:
```yaml
matrix_ma1sd_matrixorg_forwarding_enabled: true
```
## Customizing email templates
If you'd like to change the default email templates used by ma1sd, take a look at the `matrix_ma1sd_threepid_medium_email_custom_` variables
(in the `roles/matrix-ma1sd/defaults/main.yml` file.
## ma1sd-controlled Registration
To use the [Registration](https://github.com/ma1uta/ma1sd/blob/master/docs/features/registration.md) feature of ma1sd, you can make use of the following variables:
- `matrix_synapse_enable_registration` - to enable user-initiated registration in Synapse
- `matrix_synapse_enable_registration_captcha` - to validate registering users using reCAPTCHA, as described in the [enabling reCAPTCHA](configuring_captcha.md) documentation.
- `matrix_synapse_registrations_require_3pid` - to control the types of 3pid (`'email'`, `'msisdn'`) required by the Synapse server for registering
- variables prefixed with `matrix_nginx_proxy_proxy_matrix_3pid_registration_` (e.g. `matrix_nginx_proxy_proxy_matrix_3pid_registration_enabled`) - to configure the integrated nginx webserver to send registration requests to ma1sd (instead of Synapse), so it can apply its additional functionality
- `matrix_ma1sd_configuration_extension_yaml` - to configure ma1sd as required. See the [Registration feature's docs](https://github.com/ma1uta/ma1sd/blob/master/docs/features/registration.md) for inspiration. Also see the [Additional features](#additional-features) section below to learn more about how to use `matrix_ma1sd_configuration_extension_yaml`.
**Note**: For this to work, either the homeserver needs to [federate](configuring-playbook-federation.md) or the `openid` APIs need to exposed on the federation port. When federation is disabled and ma1sd is enabled, we automatically expose the `openid` APIs (only!) on the federation port. Make sure the federation port (usually `https://matrix.DOMAIN:8448`) is whitelisted in your firewall (even if you don't actually use/need federation).
## Authentication
[Authentication](https://github.com/ma1uta/ma1sd/blob/master/docs/features/authentication.md) provides the possibility to use your own [Identity Stores](https://github.com/ma1uta/ma1sd/blob/master/docs/stores/README.md) (for example LDAP) to authenticate users on your Homeserver. The following configuration can be used to authenticate against an LDAP server:
What this playbook configures for your is some bare minimum Identity Server functionality, so that you won't need to rely on external 3rd party services.
A few variables can be toggled in this playbook to alter the ma1sd configuration that gets generated.
Still, ma1sd can do much more.
You can refer to the [ma1sd website](https://github.com/ma1uta/ma1sd) for more details and configuration options.
To use a more custom configuration, you can define a `matrix_ma1sd_configuration_extension_yaml` string variable
and put your configuration in it.
To learn more about how to do this, refer to the information about `matrix_ma1sd_configuration_extension_yaml` in the [default variables file](../roles/matrix-ma1sd/defaults/main.yml) of the ma1sd component.
## Example: SMS verification
If your use case requires mobile verification, it is quite simple to integrate ma1sd with [Twilio](https://www.twilio.com/), an online telephony services gateway. Their prices are reasonable for low-volume projects and integration can be done with the following configuration:
```yaml
matrix_ma1sd_configuration_extension_yaml: |
threepid:
medium:
msisdn:
connectors:
twilio:
account_sid: '<secret-SID>'
auth_token: '<secret-token>'
number: '+<msisdn-number>'
```
## Example: Open Registration for every Domain
If you want to open registration for any domain, you have to setup the allowed domains with ma1sd's `blacklist` and `whitelist`. The default behavior when neither the `blacklist`, nor the `whitelist` match, is to allow registration. Beware: you can't block toplevel domains (aka `.xy`) because the internal architecture of ma1sd doesn't allow that.
```yaml
matrix_ma1sd_configuration_extension_yaml: |
register:
policy:
allowed: true
threepid:
email:
domain:
blacklist: ~
whitelist: ~
```
## Troubleshooting
If email address validation emails sent by ma1sd are not reaching you, you should look into [Adjusting email-sending settings](configuring-playbook-email.md).
If you'd like additional logging information, temporarily enable verbose logging for ma1sd.
Example configuration (`inventory/host_vars/matrix.DOMAIN/vars.yml`):
**WARNING**: This is an advanced feature! It requires prior experience with Matrix and a specific need for using [Matrix Corporal](https://github.com/devture/matrix-corporal). If you're unsure whether you have such a need, you most likely don't.
-------------------------------------
The playbook can install and configure [matrix-corporal](https://github.com/devture/matrix-corporal) for you.
In short, it's a sort of automation and firewalling service, which is helpful if you're instaling Matrix services in a controlled corporate environment.
See that project's documentation to learn what it does and why it might be useful to you.
If you decide that you'd like to let this playbook install it for you, you'd need to also:
- (required) [set up the Shared Secret Auth password provider module](configuring-playbook-shared-secret-auth.md)
- (optional, but encouraged) [set up the REST authentication password provider module](configuring-playbook-rest-auth.md)
## Playbook configuration
You would then need some configuration like this:
```yaml
# The Shared Secret Auth password provider module is required for Corporal to work.
# Because Corporal peridoically performs lots of user logins from the same IP,
# you may need raise Synapse's ratelimits.
# The values below are just an example. Tweak to your use-case (number of users, etc.)
matrix_synapse_rc_login:
address:
per_second: 50
burst_count: 300
account:
per_second: 0.17
burst_count: 3
failed_attempts:
per_second: 0.17
burst_count: 3
```
Matrix Corporal operates with a specific Matrix user on your server.
By default, it's `matrix-corporal` (controllable by the `matrix_corporal_reconciliation_user_id_local_part` setting, see above).
No matter what Matrix user id you configure to run it with, make sure that:
- the Matrix Corporal user is created by [registering it](registering-users.md). Use a password you remember, as you'll need to log in from time to time to create or join rooms
- the Matrix Corporal user is joined and has Admin/Moderator-level access to any rooms you want it to manage
## Matrix Corporal files
The following local filesystem paths are mounted in the `matrix-corporal` container and can be used in your configuration (or policy):
- `/matrix/corporal/config` is mounted at `/etc/matrix-corporal` (read-only)
- `/matrix/corporal/var` is mounted at `/var/matrix-corporal` (read and write)
- `/matrix/corporal/cache` is mounted at `/var/cache/matrix-corporal` (read and write)
As an example: you can create your own configuration files in `/matrix/corporal/config` and they will appear in `/etc/matrix-corporal` in the Docker container. Your configuration (stuff in `matrix_corporal_policy_provider_config`) needs to refer to these files via the local container paths - `/etc/matrix-corporal` (read-only), `/var/matrix-corporal` (read and write), `/var/cache/matrix-corporal` (read and write).
The playbook can install and configure [matrix-registration](https://github.com/ZerataX/matrix-registration) for you.
> matrix-registration is a simple python application to have a token based matrix registration.
Use matrix-registration to **create unique registration links**, which people can use to register on your Matrix server. It allows you to **keep your server's registration closed (private)**, but still allow certain people (these having a special link) to register a user account.
**matrix-registration** provides 2 things:
- **an API for creating registration tokens** (unique registration links). This API can be used via `curl` or via the playbook (see [Usage](#usage) below)
- **a user registration page**, where people can use these registration tokens. By default, exposed at `https://matrix.DOMAIN/matrix-registration`
## Installing
Adjust your playbook configuration (your `inventory/host_vars/matrix.DOMAIN/vars.yml` file):
```yaml
matrix_registration_enabled: true
# Generate a strong secret using: `pwgen -s 64 1`.
**matrix-registration** gets exposed at `https://matrix.DOMAIN/matrix-registration`
It provides various [APIs](https://github.com/ZerataX/matrix-registration/wiki/api) - for creating registration tokens, listing tokens, disabling tokens, etc. To make use of all of its capabilities, consider using `curl`.
We make the most common APIs easy to use via the playbook (see below).
### Creating registration tokens
To **create a new user registration token (link)**, use this command:
```bash
ansible-playbook -i inventory/hosts setup.yml \
--tags=generate-matrix-registration-token \
--extra-vars="one_time=yes ex_date=2021-12-31"
```
The above command creates and returns a **one-time use** token, which **expires** on the 31st of December 2021.
Adjust the `one_time` and `ex_date` variables as you see fit.
Share the unique registration link (generated by the command above) with users to let them register on your Matrix server.
### Listing registration tokens
To **list the existing user registration tokens**, use this command:
By default, this playbook installs its own nginx webserver (in a Docker container) which listens on ports 80 and 443.
If that's alright, you can skip this.
## Using Nginx status
This will serve a statuspage to the hosting machine only. Useful for monitoring software like [longview](https://www.linode.com/docs/platform/longview/longview-app-for-nginx/)
By default, if ```matrix_nginx_proxy_nginx_status_enabled``` is enabled, access to the status page would be allowed from the local IP address of the server. If you wish to allow access from other IP addresses, you can provide them as a list:
You can adjust how the SSL is served by the nginx server using the `matrix_nginx_proxy_ssl_preset` variable. We support a few presets, based on the Mozilla Server Side TLS
Recommended configurations. These presets influence the TLS Protocol, the SSL Cipher Suites and the `ssl_prefer_server_ciphers` variable of nginx.
Possible values are:
- `"modern"` - For Modern clients that support TLS 1.3, with no need for backwards compatibility
- `"intermediate"` (**default**) - Recommended configuration for a general-purpose server
- `"old"` - Services accessed by very old clients or libraries, such as Internet Explorer 8 (Windows XP), Java 6, or OpenSSL 0.9.8
**Be really carefull when setting it to `"modern"`**. This could break comunication with other Matrix servers, limiting your federation posibilities.
Besides changing the preset (`matrix_nginx_proxy_ssl_preset`), you can also directly override these 3 variables:
- `matrix_nginx_proxy_ssl_protocols`: for specifying the supported TLS protocols.
- `matrix_nginx_proxy_ssl_prefer_server_ciphers`: for specifying if the server or the client choice when negotiating the cipher. It can set to `on` or `off`.
- `matrix_nginx_proxy_ssl_ciphers`: for specifying the SSL Cipher suites used by nginx.
For more information about these variables, check the `roles/matrix-nginx-proxy/defaults/main.yml` file.
## Synapse + OpenID Connect for Single-Sign-On
If you want to use OpenID Connect as an SSO provider (as per the [Synapse OpenID docs](https://github.com/matrix-org/synapse/blob/develop/docs/openid.md)), you need to use the following configuration (in your `vars.yml` file) to instruct nginx to forward `/_synapse/oidc` to Synapse:
This playbook also allows for additional configuration to be applied to the nginx server.
If you want this playbook to obtain and renew certificates for other domains, then you can set the `matrix_ssl_additional_domains_to_obtain_certificates_for` variable (as mentioned in the [Obtaining SSL certificates for additional domains](configuring-playbook-ssl-certificates.md#obtaining-ssl-certificates-for-additional-domains) documentation as well). Make sure that you have set the DNS configuration for the domains you want to include to point at your server.
# Using your own webserver, instead of this playbook's nginx proxy (optional, advanced)
By default, this playbook installs its own nginx webserver (in a Docker container) which listens on ports 80 and 443.
If that's alright, you can skip this.
If you don't want this playbook's nginx webserver to take over your server's 80/443 ports like that,
and you'd like to use your own webserver (be it nginx, Apache, Varnish Cache, etc.), you can.
There are **2 ways you can go about it**, if you'd like to use your own webserver:
- [Method 1: Disabling the integrated nginx reverse-proxy webserver](#method-1-disabling-the-integrated-nginx-reverse-proxy-webserver)
- [Method 2: Fronting the integrated nginx reverse-proxy webserver with another reverse-proxy](#method-2-fronting-the-integrated-nginx-reverse-proxy-webserver-with-another-reverse-proxy)
## Method 1: Disabling the integrated nginx reverse-proxy webserver
This method is about completely disabling the integrated nginx reverse-proxy webserver and replicating its behavior using another webserver.
For an alternative, make sure to check Method #2 as well.
### Preparation
No matter which external webserver you decide to go with, you'll need to:
1) Make sure your web server user (something like `http`, `apache`, `www-data`, `nginx`) is part of the `matrix` group. You should run something like this: `usermod -a -G matrix nginx`. This allows your webserver user to access files owned by the `matrix` group. When using an external nginx webserver, this allows it to read configuration files from `/matrix/nginx-proxy/conf.d`. When using another server, it would make other files, such as `/matrix/static-files/.well-known`, accessible to it.
2) Edit your configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`) to disable the integrated nginx server:
```yaml
matrix_nginx_proxy_enabled: false
```
3) **If you'll manage SSL certificates by yourself**, edit your configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`) to disable SSL certificate retrieval:
```yaml
matrix_ssl_retrieval_method: none
```
**Note**: During [installation](installing.md), unless you've disabled SSL certificate management (`matrix_ssl_retrieval_method: none`), the playbook would need 80 to be available, in order to retrieve SSL certificates. **Please manually stop your other webserver while installing**. You can start it back up afterwards.
### Using your own external nginx webserver
Once you've followed the [Preparation](#preparation) guide above, it's time to set up your external nginx server.
Even with `matrix_nginx_proxy_enabled: false`, the playbook still generates some helpful files for you in `/matrix/nginx-proxy/conf.d`.
Those configuration files are adapted for use with an external web server (one not running in the container network).
You can most likely directly use the config files installed by this playbook at: `/matrix/nginx-proxy/conf.d`. Just include them in your own `nginx.conf` like this: `include /matrix/nginx-proxy/conf.d/*.conf;`
Note that if your nginx version is old, it might not like our default choice of SSL protocols (particularly the fact that the brand new `TLSv1.3` protocol is enabled). You can override the protocol list by redefining the `matrix_nginx_proxy_ssl_protocols` variable. Example:
```yaml
# Custom protocol list (removing `TLSv1.3`) to suit your nginx version.
matrix_nginx_proxy_ssl_protocols: "TLSv1.2"
```
If you are experiencing issues, try updating to a newer version of Nginx. As a data point in May 2021 a user reported that Nginx 1.14.2 was not working for them. They were getting errors about socket leaks. Updating to Nginx 1.19 fixed their issue.
### Using your own external Apache webserver
Once you've followed the [Preparation](#preparation) guide above, you can take a look at the [examples/apache](../examples/apache) directory for a sample configuration.
### Using your own external caddy webserver
After following the [Preparation](#preparation) guide above, you can take a look at the [examples/caddy](../examples/caddy) directory for a sample configuration.
### Using your own HAproxy reverse proxy
After following the [Preparation](#preparation) guide above, you can take a look at the [examples/haproxy](../examples/haproxy) directory for a sample configuration. In this case HAproxy is used as a reverse proxy and a simple Nginx container is used to serve statically `.well-known` files.
### Using another external webserver
Feel free to look at the [examples/apache](../examples/apache) directory, or the [template files in the matrix-nginx-proxy role](../roles/matrix-nginx-proxy/templates/conf.d/).
## Method 2: Fronting the integrated nginx reverse-proxy webserver with another reverse-proxy
This method is about leaving the integrated nginx reverse-proxy webserver be, but making it not get in the way (using up important ports, trying to retrieve SSL certificates, etc.).
If you wish to use another webserver, the integrated nginx reverse-proxy webserver usually gets in the way because it attempts to fetch SSL certificates and binds to ports 80, 443 and 8448 (if Matrix Federation is enabled).
You can disable such behavior and make the integrated nginx reverse-proxy webserver only serve traffic locally (or over a local network).
You would need some configuration like this:
```yaml
# Do not retrieve SSL certificates. This shall be managed by another webserver or other means.
matrix_ssl_retrieval_method: none
# Do not try to serve HTTPS, since we have no SSL certificates.
# Disabling this also means services will be served on the HTTP port
# Coturn relies on SSL certificates that have already been obtained.
# Since we don't obtain any certificates (`matrix_ssl_retrieval_method: none` above), it won't work by default.
# An alternative is to tweak some of: `matrix_coturn_tls_enabled`, `matrix_coturn_tls_cert_path` and `matrix_coturn_tls_key_path`.
matrix_coturn_enabled: false
# Trust the reverse proxy to send the correct `X-Forwarded-Proto` header as it is handling the SSL connection.
matrix_nginx_proxy_trust_forwarded_proto: true
```
With this, nginx would still be in use, but it would not bother with anything SSL related or with taking up public ports.
All services would be served locally on `127.0.0.1:81` and `127.0.0.1:8449` (as per the example configuration above).
You can then set up another reverse-proxy server on ports 80/443/8448 for all of the expected domains and make traffic go to these local ports.
The expected domains vary depending on the services you have enabled (`matrix.DOMAIN` for sure; `element.DOMAIN`, `dimension.DOMAIN` and `jitsi.DOMAIN` are optional).
### Sample configuration for running behind Traefik 2.0
Below is a sample configuration for using this playbook with a [Traefik](https://traefik.io/) 2.0 reverse proxy.
```yaml
# Disable generation and retrieval of SSL certs
matrix_ssl_retrieval_method: none
# Configure Nginx to only use plain HTTP
matrix_nginx_proxy_https_enabled: false
# Don't bind any HTTP or federation port to the host
# (Traefik will proxy directly into the containers)
This method uses labels attached to the Nginx and Synapse containers to provide the Traefik Docker provider with the information it needs to proxy `matrix.DOMAIN`, `element.DOMAIN`, `dimension.DOMAIN` and `jitsi.DOMAIN`. Some [static configuration](https://docs.traefik.io/v2.0/reference/static-configuration/file/) is required in Traefik; namely, having endpoints on ports 443 and 8448 and having a certificate resolver.
Note that this configuration on its own does **not** redirect traffic on port 80 (plain HTTP) to port 443 for HTTPS, which may cause some issues, since the built-in Nginx proxy usually does this. If you are not already doing this in Traefik, it can be added to Traefik in a [file provider](https://docs.traefik.io/v2.0/providers/file/) as follows:
```toml
[http]
[http.routers]
[http.routers.redirect-http]
entrypoints = ["web"] # The 'web' entrypoint must bind to port 80
rule = "HostRegexp(`{host:.+}`)" # Change if you don't want to redirect all hosts to HTTPS
service = "dummy" # Unused, but all routers need services (for now)
middlewares = ["https"]
[http.services]
[http.services.dummy.loadbalancer]
[[http.services.dummy.loadbalancer.servers]]
url = "localhost"
[http.middlewares]
[http.middlewares.https.redirectscheme]
scheme = "https"
permanent = true
```
You can use the following `docker-compose.yml` as example to launch Traefik.
|`matrix_postgres_backup_enabled`|`false`|Set to true to use [docker-postgres-backup-local](https://github.com/prodrigestivill/docker-postgres-backup-local) to create automatic database backups|
|`matrix_postgres_backup_schedule`| `'@daily'` |Cron-schedule specifying the interval between postgres backups.|
|`matrix_postgres_backup_keep_days`|`7`|Number of daily backups to keep|
|`matrix_postgres_backup_keep_weeks`|`4`|Number of weekly backups to keep|
|`matrix_postgres_backup_keep_months`|`12`|Number of monthly backups to keep|
|`matrix_postgres_backup_path` | `"{{ matrix_base_data_path }}/postgres-backup"` | Storagepath for the database backups|
## Installing
After configuring the playbook, run the [installation](installing.md) command again:
By default, a [Grafana](https://grafana.com/) web user-interface will be available at `https://stats.<your-domain>`.
## What does it do?
Name | Description
-----|----------
`matrix_prometheus_enabled`|[Prometheus](https://prometheus.io) is a time series database. It holds all the data we're going to talk about.
`matrix_prometheus_node_exporter_enabled`|[Node Exporter](https://prometheus.io/docs/guides/node-exporter/) is an addon of sorts to Prometheus that collects generic system information such as CPU, memory, filesystem, and even system temperatures
`matrix_grafana_enabled`|[Grafana](https://grafana.com/) is the visual component. It shows (on the `stats.<your-domain>` subdomain) the dashboards with the graphs that we're interested in
`matrix_grafana_anonymous_access`|By default you need to log in to see graphs. If you want to publicly share your graphs (e.g. when asking for help in [`#synapse:matrix.org`](https://matrix.to/#/#synapse:matrix.org?via=matrix.org&via=privacytools.io&via=mozilla.org)) you'll want to enable this option.
`matrix_grafana_default_admin_user`<br>`matrix_grafana_default_admin_password`|By default Grafana creates a user with `admin` as the username and password. If you feel this is insecure and you want to change it beforehand, you can do that here
## Security and privacy
Metrics and resulting graphs can contain a lot of information. This includes system specs but also usage patterns. This applies especially to small personal/family scale homeservers. Someone might be able to figure out when you wake up and go to sleep by looking at the graphs over time. Think about this before enabling anonymous access. And you should really not forget to change your Grafana password.
Most of our docker containers run with limited system access, but the `prometheus-node-exporter` has access to the host network stack and (readonly) root filesystem. This is required to report on them. If you don't like that, you can set `matrix_prometheus_node_exporter_enabled: false` (which is actually the default). You will still get Synapse metrics with this container disabled. Both of the dashboards will always be enabled, so you can still look at historical data after disabling either source.
## Collecting metrics to an external Prometheus server
If you wish, you could expose homeserver metrics without enabling (installing) Prometheus and Grafana via the playbook. This may be useful for hooking Matrix services to an external Prometheus/Grafana installation.
To do this, you may be interested in the following variables:
Name | Description
-----|----------
`matrix_synapse_metrics_enabled`|Set this to `true` to make Synapse expose metrics (locally, on the container network)
`matrix_nginx_proxy_proxy_synapse_metrics`|Set this to `true` to make matrix-nginx-proxy expose the Synapse metrics at `https://matrix.DOMAIN/_synapse/metrics`
`matrix_nginx_proxy_proxy_synapse_metrics_basic_auth_enabled`|Set this to `true` to password-protect (using HTTP Basic Auth) `https://matrix.DOMAIN/_synapse/metrics` (the username is always `prometheus`, the password is defined in `matrix_nginx_proxy_proxy_synapse_metrics_basic_auth_key`)
`matrix_nginx_proxy_proxy_synapse_metrics_basic_auth_key`|Set this to a password to use for HTTP Basic Auth for protecting `https://matrix.DOMAIN/_synapse/metrics` (the username is always `prometheus` - it's not configurable)
`matrix_server_fqn_grafana`|Use this variable to override the domain at which the Grafana web user-interface is at (defaults to `stats.DOMAIN`)
### Collecting system and Postgres metrics to an external Prometheus server (advanced)
When you normally enable the Prometheus and Grafana via the playbook, it will also show general system (via node-exporter) and Postgres (via postgres-exporter) stats. If you are instead collecting your metrics to an external Prometheus server, you can follow this advanced configuration example to also export these stats.
It would be possible to use `matrix_prometheus_node_exporter_container_http_host_bind_port` etc., but that is not always the best choice, for example because your server is on a public network.
Use the following variables in addition to the ones mentioned above:
Name | Description
-----|----------
`matrix_nginx_proxy_proxy_grafana_enabled`|Set this to `true` to make the stats subdomain (`matrix_server_fqn_grafana`) available via the Nginx proxy
`matrix_ssl_additional_domains_to_obtain_certificates_for`|Add `"{{ matrix_server_fqn_grafana }}"` to this list to have letsencrypt fetch a certificate for the stats subdomain
`matrix_prometheus_node_exporter_enabled`|Set this to `true` to enable the node (general system stats) exporter
`matrix_prometheus_postgres_exporter_enabled`|Set this to `true` to enable the Postgres exporter
`matrix_nginx_proxy_proxy_grafana_additional_server_configuration_blocks`|Add locations to this list depending on which of the above exporters you enabled (see below)
You can customize the `location`s to your liking, just point your Prometheus to there later (e.g. `stats.DOMAIN/node-exporter/metrics`). Nginx is very picky about the `proxy_pass`syntax: take care to follow the example closely and note the trailing slash as well as absent use of variables. postgres-exporter uses the nonstandard port 9187.
## More information
- [Understanding Synapse Performance Issues Through Grafana Graphs](https://github.com/matrix-org/synapse/wiki/Understanding-Synapse-Performance-Issues-Through-Grafana-Graphs) at the Synapse Github Wiki
- [The Prometheus scraping rules](https://github.com/matrix-org/synapse/tree/master/contrib/prometheus) (we use v2)
# Enabling metrics and graphs for Postgres (optional)
Expanding on the metrics exposed by the [synapse exporter and the node exporter](configuring-playbook-prometheus-grafana.md), the playbook enables the [postgres exporter](https://github.com/prometheus-community/postgres_exporter) that exposes more detailed information about what's happening on your postgres database.
You can enable this with the following settings in your configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`):
```yaml
matrix_prometheus_postgres_exporter_enabled: true
# the role creates a postgres user as credential. You can configure these if required:
`matrix_prometheus_postgres_exporter_enabled`|Enable the postgres prometheus exporter. This sets up the docker container, connects it to the database and adds a 'job' to the prometheus config which tells prometheus about this new exporter. The default is 'false'
`matrix_prometheus_postgres_exporter_database_username`| The 'username' for the user that the exporter uses to connect to the database. The default is 'matrix_prometheus_postgres_exporter'
`matrix_prometheus_postgres_exporter_database_password`| The 'password' for the user that the exporter uses to connect to the database.
If you wish for users to **authenticate only against configured password providers** (like this one), **without consulting Synapse's local database**, feel free to disable it:
By default, this playbook **used to install** the [Riot-web](https://github.com/vector-im/riot-web) Matrix client web application.
Riot has since been [renamed to Element](https://element.io/blog/welcome-to-element/).
- to learn more about Element and its configuration, see our dedicated [Configuring Element](configuring-playbook-client-element.md) documentation page
- to learn how to migrate from Riot to Element, see [Migrating to Element](#migrating-to-element) below
## Migrating to Element
### Migrating your custom settings
If you have custom `matrix_riot_web_` variables in your `inventory/host_vars/matrix.DOMAIN/vars.yml` file, you'll need to rename them (`matrix_riot_web_` -> `matrix_client_element_`).
Some other playbook variables (but not all) with `riot` in their name are also renamed. The playbook checks and warns if you are using the old name for some commonly used ones.
### Domain migration
We used to set up Riot at the `riot.DOMAIN` domain. The playbook now sets up Element at `element.DOMAIN` by default.
There are a few options for handling this:
- (**avoiding changes** - using the old `riot.DOMAIN` domain and avoiding DNS changes) -- to keep using `riot.DOMAIN` instead of `element.DOMAIN`, override the domain at which the playbook serves Element: `matrix_server_fqn_element: "riot.{{ matrix_domain }}"`
- (**embracing changes** - using only `element.DOMAIN`) - set up the `element.DOMAIN` DNS record (see [Configuring DNS](configuring-dns.md)). You can drop the `riot.DOMAIN` in this case. If so, you may also wish to remove old SSL certificates (`rm -rf /matrix/ssl/config/live/riot.DOMAIN`) and renewal configuration (`rm -f /matrix/ssl/config/renewal/riot.DOMAIN.conf`), so that `certbot` would stop trying to renew them.
- (**embracing changes and transitioning smoothly** - using both `element.DOMAIN` and `riot.DOMAIN`) - to serve Element at the new domain (`element.DOMAIN`) and to also have `riot.DOMAIN` redirect there - set up the `element.DOMAIN` DNS record (see [Configuring DNS](configuring-dns.md)) and enable Riot to Element redirection (`matrix_nginx_proxy_proxy_riot_compat_redirect_enabled: true`).
### Re-running the playbook
As always, after making the necessary DNS and configuration adjustments, re-run the playbook to apply the changes:
# Storing Matrix media files on Amazon S3 (optional)
By default, this playbook configures your server to store Synapse's content repository (`media_store`) files on the local filesystem.
If that's alright, you can skip this.
If you'd like to store Synapse's content repository (`media_store`) files on Amazon S3 (or other S3-compatible service),
you can let this playbook configure [Goofys](https://github.com/kahing/goofys) for you.
Using a Goofys-backed media store works, but performance may not be ideal. If possible, try to use a region which is close to your Matrix server.
If you'd like to move your locally-stored media store data to Amazon S3 (or another S3-compatible object store), we also provide some migration instructions below.
## Amazon S3
You'll need an Amazon S3 bucket and some IAM user credentials (access key + secret key) with full write access to the bucket. Example security policy:
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1400105486000",
"Effect": "Allow",
"Action": [
"s3:*"
],
"Resource": [
"arn:aws:s3:::your-bucket-name",
"arn:aws:s3:::your-bucket-name/*"
]
}
]
}
```
You then need to enable S3 support in your configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`).
matrix_s3_media_store_custom_endpoint: "https://s3.us-west-002.backblazeb2.com" # this may be different for your bucket
```
If you have local media store files and wish to migrate to Backblaze B2 subsequently, follow our [migration guide to Backblaze B2](#migrating-to-backblaze-b2) below instead of applying this configuration as-is.
## Migrating from local filesystem storage to S3
It's a good idea to [make a complete server backup](faq.md#how-do-i-backup-the-data-on-my-server) before migrating your local media store to an S3-backed one.
Follow one of the guides below for a migration path from a locally-stored media store to one stored on S3-compatible storage:
- [Migrating to any S3-compatible storage (universal, but likely slow)](#migrating-to-any-s3-compatible-storage-universal-but-likely-slow)
- [Migrating to Backblaze B2](#migrating-to-backblaze-b2)
### Migrating to any S3-compatible storage (universal, but likely slow)
It's a good idea to [make a complete server backup](faq.md#how-do-i-backup-the-data-on-my-server) before doing this.
1. Proceed with the steps below without stopping Matrix services
2. Start by adding the base S3 configuration in your `vars.yml` file (seen above, may be different depending on the S3 provider of your choice)
3. In addition to the base configuration you see above, add this to your `vars.yml` file:
This enables S3 support, but mounts the S3 storage bucket to `/matrix/s3-media-store` without hooking it to your homeserver yet. Your homeserver will still continue using your local filesystem for its media store.
5. Run the playbook to apply the changes: `ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,start`
6. Do an **initial sync of your files** by running this **on the server** (it may take a very long time):
# Setting up the Shared Secret Auth password provider module (optional, advanced)
The playbook can install and configure [matrix-synapse-shared-secret-auth](https://github.com/devture/matrix-synapse-shared-secret-auth) for you.
See that project's documentation to learn what it does and why it might be useful to you.
If you decide that you'd like to let this playbook install it for you, you need some configuration (`inventory/host_vars/matrix.<your-domain>/vars.yml`) like this:
You can generate a strong shared secret with a command like this: `pwgen -s 64 1`
## Authenticating only using a password provider
If you wish for users to **authenticate only against configured password providers** (like this one), **without consulting Synapse's local database**, feel free to disable it:
By default, this playbook retrieves and auto-renews free SSL certificates from [Let's Encrypt](https://letsencrypt.org/) for the domains it needs (`matrix.<your-domain>` and possibly `element.<your-domain>`)
Those certificates are used when configuring the nginx reverse proxy installed by this playbook.
They can also be used for configuring [your own webserver](configuring-playbook-own-webserver.md), in case you're not using the integrated nginx server provided by the playbook.
If you need to retrieve certificates for other domains (e.g. your base domain) or more control over certificate retrieval, read below.
Things discussed in this document:
- [Using self-signed SSL certificates](#using-self-signed-ssl-certificates), if you can't use Let's Encrypt or just need a test setup
- [Using your own SSL certificates](#using-your-own-ssl-certificates), if you don't want to or can't use Let's Encrypt certificates, but are still interested in using the integrated nginx reverse proxy server
- [Not bothering with SSL certificates](#not-bothering-with-ssl-certificates), if you're using [your own webserver](configuring-playbook-own-webserver.md) and would rather this playbook leaves SSL certificate management to you
- [Obtaining SSL certificates for additional domains](#obtaining-ssl-certificates-for-additional-domains), if you'd like to host additional domains on the Matrix server and would like the playbook to help you obtain and renew certificates for those domains automatically
## Using self-signed SSL certificates
For private deployments (not publicly accessible from the internet), you may not be able to use Let's Encrypt certificates.
If self-signed certificates are alright with you, you can ask the playbook to generate such for you with the following configuration:
```yaml
matrix_ssl_retrieval_method: self-signed
```
If you get a `Cannot reach homeserver` error in Element, you will have to visit `https://matrix.<your-domain>` in your browser and agree to the certificate exception before you can login.
## Using your own SSL certificates
If you'd like to manage SSL certificates by yourself and have the playbook use your certificate files, you can use the following configuration:
```yaml
matrix_ssl_retrieval_method: manually-managed
```
With such a configuration, the playbook would expect you to drop the SSL certificate files in the directory specified by `matrix_ssl_config_dir_path` (`/matrix/ssl/config` by default) obeying the following hierarchy:
where `<domain>` refers to the domains that you need (usually `matrix.<your-domain>` and `element.<your-domain>`).
## Not bothering with SSL certificates
If you're [using an external web server](configuring-playbook-own-webserver.md) which is not nginx, or you would otherwise want to manage its certificates without this playbook getting in the way, you can completely disable SSL certificate management with the following configuration:
```yaml
matrix_ssl_retrieval_method: none
```
With such a configuration, no certificates will be retrieved at all. You're free to manage them however you want.
## Obtaining SSL certificates for additional domains
The playbook tries to be smart about the certificates it will obtain for you.
- possibly for `element.<your-domain>`, unless you have disabled the [Element client component](configuring-playbook-client-element.md) using `matrix_client_element_enabled: false`
- possibly for `riot.<your-domain>`, if you have explicitly enabled Riot to Element redirection (for background compatibility) using `matrix_nginx_proxy_proxy_riot_compat_redirect_enabled: true`
- possibly for `dimension.<your-domain>`, if you have explicitly [set up Dimension](configuring-playbook-dimension.md).
- possibly for `jitsi.<your-domain>`, if you have explicitly [set up Jitsi](configuring-playbook-jitsi.md).
- possibly for your base domain (`<your-domain>`), if you have explicitly configured [Serving the base domain](configuring-playbook-base-domain-serving.md)
If you are hosting other domains on the Matrix machine, you can make the playbook obtain and renew certificates for those other domains too.
To do that, simply define your own custom configuration like this:
```yaml
# In this example, we retrieve 2 extra certificates,
# one for the base domain (in the `matrix_domain` variable) and one for a hardcoded domain.
# Adding any other additional domains (hosted on the same machine) is possible.
After redefining `matrix_ssl_domains_to_obtain_certificates_for`, to actually obtain certificates you should:
- make sure the web server occupying port 80 is stopped. If you are using matrix-nginx-proxy server (which is the default for this playbook), you need to stop it temporarily by running `systemctl stop matrix-nginx-proxy` on the server.
- re-run the SSL part of the playbook and restart all services: `ansible-playbook -i inventory/hosts setup.yml --tags=setup-ssl,start`
The certificate files would be made available in `/matrix/ssl/config/live/<your-other-domain>/...`.
For automated certificate renewal to work, each port `80` vhost for each domain you are obtaining certificates for needs to forward requests for `/.well-known/acme-challenge` to the certbot container we use for renewal.
See how this is configured for the `matrix.` subdomain in `/matrix/nginx-proxy/conf.d/matrix-synapse.conf`
Don't be alarmed if the above configuration file says port `8080`, instead of port `80`. It's due to port mapping due to our use of containers.
The playbook can install and configure the [Sygnal](https://github.com/matrix-org/sygnal) push gateway for you.
See the project's [documentation](https://github.com/matrix-org/sygnal) to learn what it does and why it might be useful to you.
**Note**: most people don't need to install their own gateway. As Sygnal's [Notes for application developers](https://github.com/matrix-org/sygnal/blob/master/docs/applications.md) documentation says:
> It is not feasible to allow end-users to configure their own Sygnal instance, because the Sygnal instance needs the appropriate FCM or APNs secrets that belong to the application.
This optional playbook component is only useful to people who develop/build their own Matrix client applications themselves.
## Adjusting the playbook configuration
Add the following configuration to your `inventory/host_vars/matrix.DOMAIN/vars.yml` file (adapt to your needs):
```yaml
matrix_sygnal_enabled: true
# You need at least 1 app defined.
# The configuration below is incomplete. Read more below.
matrix_sygnal_apps:
com.example.myapp.ios:
type: apns
keyfile: /data/my_key.p8
# .. more configuration ..
com.example.myapp.android:
type: gcm
api_key: your_api_key_for_gcm
# .. more configuration ..
matrix_aux_file_definitions:
- dest: "{{ matrix_sygnal_data_path }}/my_key.p8"
content: |
some
content
here
mode: '0600'
owner: "{{ matrix_user_username }}"
group: "{{ matrix_user_groupname }}"
```
For a more complete example of available fields and values they can take, see `roles/matrix-sygnal/templates/sygnal.yaml.j2` (or the [upstream `sygnal.yaml.sample` configuration file](https://github.com/matrix-org/sygnal/blob/master/sygnal.yaml.sample)).
Configuring [GCM/FCM](https://firebase.google.com/docs/cloud-messaging/) is easier, as it only requires that you provide some config values.
To configure [APNS](https://developer.apple.com/notifications/) (Apple Push Notification Service), you'd need to provide one or more certificate files.
To do that, the above example configuration:
- makes use of the `matrix-aux` role (and its `matrix_aux_file_definitions` variable) to make the playbook install files into `/matrix/sygnal/data` (the `matrix_sygnal_data_path` variable). See `roles/matrix-aux/defaults/main.yml` for usage examples. It also makes sure the files are owned by `matrix:matrix`, so that Sygnal can read them. Of course, you can also install these files manually yourself, if you'd rather not use `matrix-aux`.
- references these files in the Sygnal configuration (`matrix_sygnal_apps`) using a path like `/data/..` (the `/matrix/sygnal/data` directory on the host system is mounted into the `/data` directory inside the container)
## Installing
Don't forget to add `sygnal.<your-domain>` to DNS as described in [Configuring DNS](configuring-dns.md) before running the playbook.
After configuring the playbook, run the [installation](installing.md) command again:
To make use of your Sygnal installation, you'd need to build your own Matrix client application, which uses the same API keys (for [GCM/FCM](https://firebase.google.com/docs/cloud-messaging/)) and certificates (for [APNS](https://developer.apple.com/notifications/)) and is also pointed to `https://sygnal.DOMAIN` as the configured push server.
Refer to Sygnal's [Notes for application developers](https://github.com/matrix-org/sygnal/blob/master/docs/applications.md) document.
The playbook can install and configure [synapse-admin](https://github.com/Awesome-Technologies/synapse-admin) for you.
It's a web UI tool you can use to **administrate users and rooms on your Matrix server**.
See the project's [documentation](https://github.com/Awesome-Technologies/synapse-admin) to learn what it does and why it might be useful to you.
## Adjusting the playbook configuration
Add the following configuration to your `inventory/host_vars/matrix.DOMAIN/vars.yml` file:
```yaml
matrix_synapse_admin_enabled: true
```
**Note**: Synapse Admin requires Synapse's [Admin APIs](https://github.com/matrix-org/synapse/tree/master/docs/admin_api) to function. Access to them is restricted with a valid access token, so exposing them publicly should not be a real security concern. Still, for additional security, we normally leave them unexposed, following [official Synapse reverse-proxying recommendations](https://github.com/matrix-org/synapse/blob/master/docs/reverse_proxy.md#synapse-administration-endpoints). Because Synapse Admin needs these APIs to function, when installing Synapse Admin, we **automatically** exposes them publicly for you (equivalent to `matrix_nginx_proxy_proxy_matrix_client_api_forwarded_location_synapse_admin_api_enabled: true`).
## Installing
After configuring the playbook, run the [installation](installing.md) command again:
By default, this playbook configures the [Synapse](https://github.com/matrix-org/synapse) Matrix server, so that it works for the general case.
If that's enough for you, you can skip this document.
The playbook provides lots of customization variables you could use to change Synapse's settings.
Their defaults are defined in [`roles/matrix-synapse/defaults/main.yml`](../roles/matrix-synapse/defaults/main.yml) and they ultimately end up in the generated `/matrix/synapse/config/homeserver.yaml` file (on the server). This file is generated from the [`roles/matrix-synapse/templates/synapse/homeserver.yaml.j2`](../roles/matrix-synapse/templates/synapse/homeserver.yaml.j2) template.
**If there's an existing variable** which controls a setting you wish to change, you can simply define that variable in your configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`) and [re-run the playbook](installing.md) to apply the changes.
Alternatively, **if there is no pre-defined variable** for a Synapse setting you wish to change:
- you can either **request a variable to be created** (or you can submit such a contribution yourself). Keep in mind that it's **probably not a good idea** to create variables for each one of Synapse's various settings that rarely get used.
- or, you can **extend and override the default configuration** ([`homeserver.yaml.j2`](../roles/matrix-synapse/templates/synapse/homeserver.yaml.j2)) by making use of the `matrix_synapse_configuration_extension_yaml` variable. You can find information about this in [`roles/matrix-synapse/defaults/main.yml`](../roles/matrix-synapse/defaults/main.yml).
- or, if extending the configuration is still not powerful enough for your needs, you can **override the configuration completely** using `matrix_synapse_configuration` (or `matrix_synapse_configuration_yaml`). You can find information about this in [`roles/matrix-synapse/defaults/main.yml`](../roles/matrix-synapse/defaults/main.yml).
## Load balancing with workers
To have Synapse gracefully handle thousands of users, worker support should be enabled. It factors out some homeserver tasks and spreads the load of incoming client and server-to-server traffic between multiple processes. More information can be found in the [official Synapse workers documentation](https://github.com/matrix-org/synapse/blob/master/docs/workers.md).
To enable Synapse worker support, update your `inventory/host_vars/matrix.DOMAIN/vars.yml` file:
```yaml
matrix_synapse_workers_enabled: true
```
We support a few configuration presets (`matrix_synapse_workers_preset: one-of-each` being the default configuration):
- `little-federation-helper` - a very minimal worker configuration to improve federation performance
- `one-of-each` - one worker of each supported type
If you'd like more customization power, you can start with one of the presets and tweak various `matrix_synapse_workers_*_count` variables manually.
If you increase worker counts too much, you may need to increase the maximum number of Postgres connections too (example):
```yaml
matrix_postgres_process_extra_arguments: [
"-c 'max_connections=200'"
]
```
If you're using the default setup (the `matrix-nginx-proxy` webserver being enabled) or you're using your own `nginx` server (which imports the configuration files generated by the playbook), you're good to go. If you use some other webserver, you may need to tweak your reverse-proxy setup manually to forward traffic to the various workers.
In case any problems occur, make sure to have a look at the [list of synapse issues about workers](https://github.com/matrix-org/synapse/issues?q=workers+in%3Atitle) and your `journalctl --unit 'matrix-*'`.
## Synapse Admin
Certain Synapse administration tasks (managing users and rooms, etc.) can be performed via a web user-interace, if you install [Synapse Admin](configuring-playbook-synapse-admin.md).
## Synapse + OpenID Connect for Single-Sign-On
If you'd like to use OpenID Connect authentication with Synapse, you'll need some additional reverse-proxy configuration (see [our nginx reverse-proxy doc page](configuring-playbook-nginx.md#synapse-openid-connect-for-single-sign-on)).
In case you encounter errors regarding the parsing of the variables, you can try to add `{% raw %}` and `{% endraw %}` blocks around them. For example ;
# Enabling Telemetry for your Matrix server (optional)
By default, this playbook configures your Matrix homeserver to not send any telemetry data anywhere.
The [matrix.org](https://matrix.org) team would really appreciate it if you could help the project out by reporting
anonymized usage statistics from your homeserver. Only very [basic aggregate
data](#usage-statistics-being-submitted) (e.g. number of users) will be reported, but it helps track the
growth of the Matrix community, and helps to make Matrix a success.
## Enabling Telemetry
If you'd like to **help by enabling submission of general usage statistics** for your homeserver, add this to your configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`):
```yaml
matrix_synapse_report_stats: true
```
## Usage statistics being submitted
If statistics reporting is enabled, the information that gets submitted to the matrix.org team [according to the source code](https://github.com/matrix-org/synapse/blob/master/synapse/app/homeserver.py) is:
- your homeserver's domain name
- uptime of the homeserver program
- [Python](https://www.python.org/) version powering your homeserver
- total number of users on your home server (including bridged users)
- total number of native Matrix users on your home server
- total number of rooms on your homeserver
- total number of daily active users on your homeserver
- total number of daily active rooms on your homeserver
- total number of messages sent per day
- cache setting information
- CPU and memory statistics for the homeserver program
The playbook installs a [Coturn](https://github.com/coturn/coturn) TURN server by default, so that clients can make audio/video calls even from [NAT](https://en.wikipedia.org/wiki/Network_address_translation)-ed networks.
By default, the Synapse chat server is configured, so that it points to the Coturn TURN server installed by the playbook.
## Disabling Coturn
If, for some reason, you'd like to prevent the playbook from installing Coturn, you can use the following configuration:
```yaml
matrix_coturn_enabled: false
```
In that case, Synapse would not point to any Coturn servers and audio/video call functionality may fail.
## Using your own external Coturn server
If you'd like to use another TURN server (be it Coturn or some other one), you can configure the playbook like this:
```yaml
# Disable integrated Coturn server
matrix_coturn_enabled: false
# Point Synapse to your other Coturn server
matrix_synapse_turn_uris:
- turns:HOSTNAME_OR_IP?transport=udp
- turns:HOSTNAME_OR_IP?transport=tcp
- turn:HOSTNAME_OR_IP?transport=udp
- turn:HOSTNAME_OR_IP?transport=tcp
```
If you have or want to enable [Jitsi](configuring-playbook-jitsi.md), you might want to enable the TURN server there too.
If you do not do it, Jitsi will fall back to an upstream service.
```yaml
matrix_jitsi_web_stun_servers:
- stun:HOSTNAME_OR_IP:PORT
```
You can put multiple host/port combinations if you like.
To configure the playbook, you need to have done the following things:
- have a server where Matrix services will run
- [configured your DNS records](configuring-dns.md)
- [retrieved the playbook's source code](getting-the-playbook.md) to your computer
You can then follow these steps inside the playbook directory:
1. create a directory to hold your configuration (`mkdir inventory/host_vars/matrix.<your-domain>`)
1. copy the sample configuration file (`cp examples/vars.yml inventory/host_vars/matrix.<your-domain>/vars.yml`)
1. edit the configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`) to your liking. You may also take a look at the various `roles/ROLE_NAME_HERE/defaults/main.yml` files and see if there's something you'd like to copy over and override in your `vars.yml` configuration file.
1. copy the sample inventory hosts file (`cp examples/hosts inventory/hosts`)
1. edit the inventory hosts file (`inventory/hosts`) to your liking
For a basic Matrix installation, that's all you need.
For a more custom setup, see the [Other configuration options](#other-configuration-options) below.
When you're done with all the configuration you'd like to do, continue with [Installing](installing.md).
## Other configuration options
### Additional useful services
- [Setting up the Dimension Integration Manager](configuring-playbook-dimension.md) (optional, but recommended; after [installing](installing.md))
- [Setting up the Jitsi video-conferencing platform](configuring-playbook-jitsi.md) (optional)
- [Setting up Dynamic DNS](configuring-playbook-dynamic-dns.md) (optional)
- [Enabling metrics and graphs (Prometheus, Grafana) for your Matrix server](configuring-playbook-prometheus-grafana.md) (optional)
Service discovery is a way for the Matrix network to discover where a Matrix server is.
There are 2 types of well-known service discovery that Matrix makes use of:
- (important) **Federation Server discovery** (`/.well-known/matrix/server`) -- assists other servers in the Matrix network with finding your server. Without a proper configuration, your server will effectively not be part of the Matrix network. Learn more in [Introduction to Federation Server Discovery](#introduction-to-federation-server-discovery)
- (not that important) **Client Server discovery** (`/.well-known/matrix/client`) -- assists programs that you use to connect to your server (e.g. Element), so that they can make it more convenient for you by automatically configuring the "Homeserver URL" and "Identity Server URL" addresses. Learn more in [Introduction to Client Server Discovery](#introduction-to-client-server-discovery)
## Introduction to Federation Server Discovery
All services created by this playbook are meant to be installed on their own server (such as `matrix.<your-domain>`).
As [per the Server-Server specification](https://matrix.org/docs/spec/server_server/r0.1.0.html#server-discovery), to use a Matrix user identifier like `@<username>:<your-domain>` while hosting services on a subdomain like `matrix.<your-domain>`, the Matrix network needs to be instructed of such delegation/redirection.
Server delegation can be configured using DNS SRV records or by setting up a `/.well-known/matrix/server` file on the base domain (`<your-domain.com>`).
Both methods have their place and will continue to do so. You only need to use just one of these delegation methods.
For simplicity reasons, our setup advocates for the `/.well-known/matrix/server` method and guides you into using that.
To learn how to set up `/.well-known/matrix/server`, read the Installing section below.
## Introduction to Client Server Discovery
Client Server Service discovery lets various client programs which support it, to receive a full user id (e.g. `@username:example.com`) and determine where the Matrix server is automatically (e.g. `https://matrix.example.com`).
This lets you (and your users) easily connect to your Matrix server without having to customize connection URLs. When using client programs that support it, you won't need to point them to `https://matrix.example.com` in Custom Server options manually anymore. The connection URL would be discovered automatically from your full username.
As [per the Client-Server specification](https://matrix.org/docs/spec/client_server/r0.4.0.html#server-discovery) Matrix does Client Server service discovery using a `/.well-known/matrix/client` file hosted on the base domain (e.g. `example.com`).
However, this playbook installs your Matrix server on another domain (e.g. `matrix.example.com`) and not on the base domain (e.g. `example.com`), so it takes a little extra manual effort to set up the file.
To learn how to set it up, read the Installing section below.
## Installing well-known files on the base domain's server
To implement the two service discovery mechanisms, your base domain's server (e.g. `example.com`) needs to run an HTTPS-capable webserver.
If you don't have a server for your base domain at all, you can use the Matrix server for this.
See [Serving the base domain](configuring-playbook-base-domain-serving.md) to learn how the playbook can help you set it up.
If you decide to go this route, you don't need to read ahead in this document. When **Serving the base domain**, the playbook takes care to serve the appropriate well-known files automatically.
If you're managing the base domain by yourself somehow, you'll need to set up serving of some `/.well-known/matrix/*` files from it via HTTPS.
To make things easy for you to set up, this playbook generates and hosts 2 well-known files on the Matrix domain's server (e.g. `https://matrix.example.com/.well-known/matrix/server` and `https://matrix.example.com/.well-known/matrix/client`), even though this is the wrong place to host them.
You have 3 options when it comes to installing the files on the base domain's server:
### (Option 1): **Copying the files manually** to your base domain's server
**Hint**: Option 2 and 3 (below) are generally a better way to do this. Make sure to go with them, if possible.
All you need to do is:
- copy `/.well-known/matrix/server` and `/.well-known/matrix/client` from the Matrix server (e.g. `matrix.example.com`) to your base domain's server (`example.com`). You can find these files in the `/matrix/static-files/.well-known/matrix` directory on the Matrix server. They are also accessible on URLs like this: `https://matrix.example.com/.well-known/matrix/server` (same for `client`).
- set up the server at your base domain (e.g. `example.com`) so that it adds an extra HTTP header when serving the `/.well-known/matrix/client` file. [CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS), the `Access-Control-Allow-Origin` header should be set with a value of `*`. If you don't do this step, web-based Matrix clients (like Element) may fail to work. Setting up headers for the `/.well-known/matrix/server` file is not necessary, as this file is only consumed by non-browsers, which don't care about CORS.
This is relatively easy to do and possibly your only choice if you can only host static files from the base domain's server.
It is, however, **a little fragile**, as future updates performed by this playbook may regenerate the well-known files and you may need to notice that and copy them over again.
### (Option 2): **Serving the base domain** from the Matrix server via the playbook
If you don't need the base domain (e.g. `example.com`) for anything else (hosting a website, etc.), you can point it to the Matrix server's IP address and tell the playbook to configure it.
This is the easiest way to set up well-known serving -- letting the playbook handle the whole base domain for you (including SSL certificates, etc.). However, if you need to use the base domain for other things (such as hosting some website, etc.), going with Option 1 or Option 3 might be more suitable.
See [Serving the base domain](configuring-playbook-base-domain-serving.md) to learn how the playbook can help you set it up.
### (Option 3): **Setting up reverse-proxying** of the well-known files from the base domain's server to the Matrix server
This option is less fragile and generally better.
On the base domain's server (e.g. `example.com`), you can set up reverse-proxying, so that any access for the `/.well-known/matrix` location prefix is forwarded to the Matrix domain's server (e.g. `matrix.example.com`).
With this method, you **don't need** to add special HTTP headers for [CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS) reasons (like `Access-Control-Allow-Origin`), because your Matrix server (where requests ultimately go) will be configured by this playbook correctly.
- **replace `DOMAIN`** in the server configuration with your actual domain name
- and: to **do this for the HTTPS-enabled server block**, as that's where Matrix expects the file to be
## Confirming it works
No matter which method you've used to set up the well-known files, if you've done it correctly you should be able to see a JSON file at both of these URLs:
- `https://<domain>/.well-known/matrix/server`
- `https://<domain>/.well-known/matrix/client`
You can also check if everything is configured correctly, by [checking if services work](maintenance-checking-services.md).
This page summarizes the container ([Docker](https://www.docker.com/)) images used by the playbook when setting up your server.
We try to stick to official images (provided by their respective projects) as much as possible.
## Container images used by default
These services are enabled and used by default, but you can turn them off, if you wish.
- [matrixdotorg/synapse](https://hub.docker.com/r/matrixdotorg/synapse/) - the official [Synapse](https://github.com/matrix-org/synapse) Matrix homeserver (optional)
- [coturn/coturn](https://hub.docker.com/r/coturn/coturn/) - the [Coturn](https://github.com/coturn/coturn) STUN/TURN server (optional)
- [vectorim/element-web](https://hub.docker.com/r/vectorim/element-web/) - the [Element](https://element.io/) web client (optional)
- [ma1uta/ma1sd](https://hub.docker.com/r/ma1uta/ma1sd/) - the [ma1sd](https://github.com/ma1uta/ma1sd) Matrix Identity server (optional)
- [postgres](https://hub.docker.com/_/postgres/) - the [Postgres](https://www.postgresql.org/) database server (optional)
- [devture/exim-relay](https://hub.docker.com/r/devture/exim-relay/) - the [Exim](https://www.exim.org/) email server (optional)
- [nginx](https://hub.docker.com/_/nginx/) - the [nginx](http://nginx.org/) web server (optional)
- [certbot/certbot](https://hub.docker.com/r/certbot/certbot/) - the [certbot](https://certbot.eff.org/) tool for obtaining SSL certificates from [Let's Encrypt](https://letsencrypt.org/) (optional)
## Optional other container images we may use
These services are not part of our default installation, but can be enabled by [configuring the playbook](configuring-playbook.md) (either before the initial installation or any time later):
- [ewoutp/goofys](https://hub.docker.com/r/ewoutp/goofys/) - the [Goofys](https://github.com/kahing/goofys) Amazon [S3](https://aws.amazon.com/s3/) file-system-mounting program (optional)
- [etherpad/etherpad](https://hub.docker.com/r/etherpad/etherpad/) - the [Etherpad](https://etherpad.org) realtime collaborative text editor that can be used in a Jitsi audio/video call or integrated as a widget into Matrix chat rooms via the Dimension integration manager (optional)
- [devture/email2matrix](https://hub.docker.com/r/devture/email2matrix/) - the [Email2Matrix](https://github.com/devture/email2matrix) email server, which can relay email messages to Matrix rooms (optional)
- [devture/matrix-corporal](https://hub.docker.com/r/devture/matrix-corporal/) - [Matrix Corporal](https://github.com/devture/matrix-corporal): reconciliator and gateway for a managed Matrix server (optional)
- [zeratax/matrix-registration](https://hub.docker.com/r/devture/zeratax-matrix-registration/) - [matrix-registration](https://github.com/ZerataX/matrix-registration): a simple python application to have a token based matrix registration (optional)
- [mautrix/telegram](https://mau.dev/mautrix/telegram/container_registry) - the [mautrix-telegram](https://github.com/mautrix/telegram) bridge to [Telegram](https://telegram.org/) (optional)
- [mautrix/whatsapp](https://mau.dev/mautrix/whatsapp/container_registry) - the [mautrix-whatsapp](https://github.com/mautrix/whatsapp) bridge to [Whatsapp](https://www.whatsapp.com/) (optional)
- [mautrix/facebook](https://mau.dev/mautrix/facebook/container_registry) - the [mautrix-facebook](https://github.com/mautrix/facebook) bridge to [Facebook](https://facebook.com/) (optional)
- [mautrix/hangouts](https://mau.dev/mautrix/hangouts/container_registry) - the [mautrix-hangouts](https://github.com/mautrix/hangouts) bridge to [Google Hangouts](https://en.wikipedia.org/wiki/Google_Hangouts) (optional)
- [mautrix/googlechat](https://mau.dev/mautrix/googlechat/container_registry) - the [mautrix-googlechat](https://github.com/mautrix/googlechat) bridge to [Google Chat](https://en.wikipedia.org/wiki/Google_Chat) (optional)
- [mautrix/instagram](https://mau.dev/mautrix/instagram/container_registry) - the [mautrix-instagram](https://github.com/mautrix/instagram) bridge to [Instagram](https://instagram.com/) (optional)
- [mautrix/signal](https://mau.dev/mautrix/signal/container_registry) - the [mautrix-signal](https://github.com/mautrix/signal) bridge to [Signal](https://www.signal.org/) (optional)
- [matrixdotorg/matrix-appservice-irc](https://hub.docker.com/r/matrixdotorg/matrix-appservice-irc) - the [matrix-appservice-irc](https://github.com/matrix-org/matrix-appservice-irc) bridge to [IRC](https://wikipedia.org/wiki/Internet_Relay_Chat) (optional)
- [halfshot/matrix-appservice-discord](https://hub.docker.com/r/halfshot/matrix-appservice-discord) - the [matrix-appservice-discord](https://github.com/Half-Shot/matrix-appservice-discord) bridge to [Discord](https://discordapp.com/) (optional)
- [cadair/matrix-appservice-slack](https://hub.docker.com/r/cadair/matrix-appservice-slack) - the [matrix-appservice-slack](https://github.com/matrix-org/matrix-appservice-slack) bridge to [Slack](https://slack.com/) (optional)
- [turt2live/matrix-appservice-webhooks](https://hub.docker.com/r/turt2live/matrix-appservice-webhooks) - the [Appservice Webhooks](https://github.com/turt2live/matrix-appservice-webhooks) bridge (optional)
- [folivonet/matrix-sms-bridge](https://hub.docker.com/repository/docker/folivonet/matrix-sms-bridge) - the [matrix-sms-bridge](https://github.com/benkuly/matrix-sms-bridge) (optional)
- [sorunome/mx-puppet-skype](https://hub.docker.com/r/sorunome/mx-puppet-skype) - the [mx-puppet-skype](https://github.com/Sorunome/mx-puppet-skype) bridge to [Skype](https://www.skype.com) (optional)
- [sorunome/mx-puppet-slack](https://hub.docker.com/r/sorunome/mx-puppet-slack) - the [mx-puppet-slack](https://github.com/Sorunome/mx-puppet-slack) bridge to [Slack](https://slack.com) (optional)
- [sorunome/mx-puppet-instagram](https://hub.docker.com/r/sorunome/mx-puppet-instagram) - the [mx-puppet-instagram](https://github.com/Sorunome/mx-puppet-instagram) bridge to [Instagram](https://www.instagram.com) (optional)
- [sorunome/mx-puppet-twitter](https://hub.docker.com/r/sorunome/mx-puppet-twitter) - the [mx-puppet-twitter](https://github.com/Sorunome/mx-puppet-twitter) bridge to [Twitter](https://twitter.com) (optional)
- [sorunome/mx-puppet-discord](https://hub.docker.com/r/sorunome/mx-puppet-discord) - the [mx-puppet-discord](https://github.com/matrix-discord/mx-puppet-discord) bridge to [Discord](https://discordapp.com) (optional)
- [xangelix/mx-puppet-groupme](https://hub.docker.com/r/xangelix/mx-puppet-groupme) - the [mx-puppet-groupme](https://gitlab.com/robintown/mx-puppet-groupme) bridge to [GroupMe](https://groupme.com/) (optional)
- [icewind1991/mx-puppet-steam](https://hub.docker.com/r/icewind1991/mx-puppet-steam) - the [mx-puppet-steam](https://github.com/icewind1991/mx-puppet-steam) bridge to [Steam](https://steampowered.com) (optional)
- [turt2live/matrix-dimension](https://hub.docker.com/r/turt2live/matrix-dimension) - the [Dimension](https://dimension.t2bot.io/) integrations manager (optional)
- [jitsi/web](https://hub.docker.com/r/jitsi/web) - the [Jitsi](https://jitsi.org/) web UI (optional)
- [jitsi/jicofo](https://hub.docker.com/r/jitsi/jicofo) - the [Jitsi](https://jitsi.org/) Focus component (optional)
- [jitsi/prosody](https://hub.docker.com/r/jitsi/prosody) - the [Jitsi](https://jitsi.org/) Prosody XMPP server component (optional)
- [jitsi/jvb](https://hub.docker.com/r/jitsi/jvb) - the [Jitsi](https://jitsi.org/) Video Bridge component (optional)
- [anoa/matrix-reminder-bot](https://hub.docker.com/r/anoa/matrix-reminder-bot) - the [matrix-reminder-bot](https://github.com/anoadragon453/matrix-reminder-bot) bot for one-off & recurring reminders and alarms (optional)
- [matrixdotorg/go-neb](https://hub.docker.com/r/matrixdotorg/go-neb) - the [Go-NEB](https://github.com/matrix-org/go-neb) bot (optional)
- [matrixdotorg/mjolnir](https://hub.docker.com/r/matrixdotorg/mjolnir) - the [mjolnir](https://github.com/matrix-org/mjolnir) moderation bot (optional)
- [awesometechnologies/synapse-admin](https://hub.docker.com/r/awesometechnologies/synapse-admin) - the [synapse-admin](https://github.com/Awesome-Technologies/synapse-admin) web UI tool for administrating users and rooms on your Matrix server (optional)
- [prom/prometheus](https://hub.docker.com/r/prom/prometheus/) - [Prometheus](https://github.com/prometheus/prometheus/) is a systems and service monitoring system
- [prom/node-exporter](https://hub.docker.com/r/prom/node-exporter/) - [Prometheus Node Exporter](https://github.com/prometheus/node_exporter/) is an addon for Prometheus that gathers standard system metrics
- [grafana/grafana](https://hub.docker.com/r/grafana/grafana/) - [Grafana](https://github.com/grafana/grafana/) is a graphing tool that works well with the above two images. Our playbook also adds two dashboards for [Synapse](https://github.com/matrix-org/synapse/tree/master/contrib/grafana) and [Node Exporter](https://github.com/rfrail3/grafana-dashboards)
- [matrixdotorg/sygnal](https://hub.docker.com/r/matrixdotorg/sygnal/) - [Sygnal](https://github.com/matrix-org/sygnal) is a reference Push Gateway for Matrix
This documentation page tries to answer various Frequently Asked Questions about all things [Matrix](https://matrix.org/), with a focus on this [Ansible](https://www.ansible.com/) playbook ([What is Ansible? How does it work?](#what-is-ansible-how-does-it-work)).
This FAQ page does not intend to replace the [matrix.org FAQ](https://matrix.org/faq/) (please see that one too).
We've only started this FAQ recently, so it's still somewhat empty.
Also, we encourage you to not dig yourself into a hole by reading way too much. When you've heard enough, proceed to [Prerequisites](prerequisites.md) to get guided into installing Matrix.
## Introductory
## Where do I find more questions and answers about Matrix?
This is a Frequently Asked Questions page focused on this [Ansible](https://www.ansible.com/) playbook ([What is Ansible? How does it work?](#what-is-ansible-how-does-it-work)) for deploying a [Matrix](https://matrix.org/) server.
For a lot more generic questions and answers, see the [matrix.org FAQ](https://matrix.org/faq/).
## What is Matrix? What is Element? What is Synapse? Why are you confusing me with so many terms?
[Matrix](https://matrix.org/) is a new type of realtime communication (chat) network, the closest analogy to which is probably "email".
You don't just use the "email" protocols (SMTP, POP3, IMAP) directly though. There's a *server* somewhere which stores your data (`@gmail.com`, `@yahoo.com`, `@hotmail.com`, `@your-company.com`) and you access it by using these "email" protocols via some *client* program (Outlook, Thunderbird, some website, etc).
In the world of the Matrix chat protocol, there are various client programs. The first and currently most full-featured one is called [Element](https://element.io/) (used to be called Riot.im and Vector.im in the past). There are [many other clients](https://matrix.org/clients/). You can switch clients as much as you want until you find the one that is right for you on a given platform (you may use Element on your desktop, but Fluffychat on your phone, etc).
Matrix is also like email due to the fact that there are many servers around the world which can all talk to each other (you can send email from `@gmail.com` addresses to `@yahoo.com` and `@hotmail.com` addresses). It's the same with Matrix (`@bob:his-domain.com` can talk to `@alice:her-domain.org`).
If someone else is hosting your Matrix server (you being `@user:matrix.org` or some other public server like this), all you need is a Matrix client program, like Element.
If you'd like to host your own server (you being `@user:your-own-domain.com`), you'd need to set up a Matrix server program, like Synapse.
In short:
- Matrix is the protocol - a set of rules about how the chat network operates
- Element is a client program you can use to participate on the Matrix chat network via some server (yours or someone else's). There are also [many other client programs](https://matrix.org/clients/).
- Synapse is a server program you can use to host your very own Matrix server.
This FAQ here mostly focuses on installing various Matrix services using the Ansible automation tool. You can learn much more about Matrix in the [matrix.org FAQ](https://matrix.org/faq/).
## People I wish to talk to are not on Matrix. Can I talk to them?
You most likely can. Besides Matrix-native chats, Matrix also supports the concept of "bridging", which allows you to plug other networks into it.
This Ansible playbook can help you install [tens of bridges for various networks](configuring-playbook.md#bridging-other-networks).
Besides setting up your own bridges (preferable), you can also use some [public bridges hosted by others](https://publiclist.anchel.nl/#bridges).
## How do I get started with Matrix?
One of [Matrix](https://matrix.org/)'s distinguishing strengths (compared to other chat networks) is its decentralized nature. There's not just one entity (company, organization) controlling the servers. Rather there's thousands of servers operated by different people - one server being insecure, slow or disrespective toward its users does not affect the rest of the network. To participate in that decentralization in its fullest, consider hosting your own server or using some public server other than the largest/default one (`matrix.org`).
There are 3 ways to get into Martix, depending on your technical ability and needs:
- **using the existing default server** - the easiest way is to use an existing server. The largest public Matrix server is `matrix.org` and it's configured as a default server in clients such as [Element](https://element.io) and many others. Just use Element on the browser via that link (or download the Element app on a smartphone), create an account and start chatting.
- **using some other server** - instead of using the largest public server (`matrix.org`), you can use another public one. Here's a [list of public Matrix servers](https://publiclist.anchel.nl/) to choose from. Again, you download [Element](https://element.io) or [some other client](https://matrix.org/clients/) of your choosing and adjust the homeserver URL during login.
- **using your own server** - running your own server puts you in ultimate control of your data. It also lets you have your own user identifiers (e.g. `@bob:your-domain.com`). See [How do I set up my own Matrix server](#how-do-i-set-up-my-own-matrix-server).
### How do I set up my own Matrix server?
Normally, you'd first choose the [Matrix](https://matrix.org/) server software you'd like to run. At the time of this writing (January/2021), there's only one fully-featured server program, so there's only one reasonable choice. That's [Synapse](https://github.com/matrix-org/synapse).
There are [many guides about installing Synapse](https://matrix.org/docs/guides/#installing-synapse). Using this Ansible playbook is just one way of doing it.
Naturally, we're biased, so our usual recommendation is to go with this [Ansible](https://www.ansible.com/) playbook, instead of installing Synapse (and many many other things around it) manually.
To get started with the playbook, start at the [Prerequisites](prerequisites.md) page.
### What is Ansible? How does it work?
[Ansible](https://www.ansible.com/) is an automation program. This "playbook" is a collection of tasks/scripts that will set up a [Matrix](https://matrix.org/) server for you, so you don't have to perform these tasks manually.
We have written these automated tasks for you and all you need to do is execute them using the Ansible program.
You can install Ansible and this playbook code repository on your own computer and tell it to install Matrix services at the server living at `matrix.DOMAIN`. We recommend installing Ansible on your own computer.
Alternatively, you can download Ansible and the playbook itself directly on the `matrix.DOMAIN` server.
To learn more, see our [dedicated Ansible documentation page](ansible.md).
### Why use this playbook and not install Synapse and other things manually?
There are various guides telling you how easy it is to install [Synapse](https://github.com/matrix-org/synapse).
Reading the documentation of this Ansible playbook, you may also be thinking:
> I don't know what [Ansible](https://www.ansible.com/) is. I don't know what [Docker](https://www.docker.com/) is. This looks more complicated.
.. so you may be leaning toward [installing Synapse manually](https://github.com/matrix-org/synapse/blob/master/INSTALL.md).
The problem with a manual installation is:
- Synapse is written in Python. If not packaged for your distribution, you'd need to install various Python modules, etc., and keep them updated.
- Synapse requires a [Postgres](https://www.postgresql.org/) database (it can run on SQLite, but that's very much discouraged). So you'd need to install Postgres as well.
- you may also need a reverse-proxy server in front of it (nginx, Apache), so you'd need to be familiar with that
- SSL is required, so you'd need to obtain Let's Encrypt (or other free or non-free) certificates for one or more domain names. You'd need to be familiar with [certbot](https://certbot.eff.org/) (when using Let's Encrypt) or similar software.
- for each additional component you'd like to add (client like [Element](https://element.io), bridge to some other chat network, Integration Manager (sitckers, other services), Identity Manager, etc.), you'll need to spend extra time installing and wiring it with the rest of the system in a way that works.
- you'll likely get slower updates for all of these components, depending on your distro packaging or your own time and ability
The playbook, on the other hand, installs a bunch of components for you by default, obtains SSL certificates for you, etc. If you'd like, you can enable various bridges and other services with very little effort. All the components are wired to work together.
All services run in Docker containers (most being officially provided by each component's developers), so we're not at the mercy of distro packaging.
### Why use this playbook and not just use the Docker image directly?
Reasons are similar to the reasons for not installing manually.
Besides Synapse, you'd need other things - a Postgres database, likely the [Element](https://element.io) client, etc., etc.
Using the playbook, you get all these components in a way that works well together out of the box.
### What's different about this Ansible playbook compared to [EMnify/matrix-synapse-auto-deploy](https://github.com/EMnify/matrix-synapse-auto-deploy)?
This is similar to the [EMnify/matrix-synapse-auto-deploy](https://github.com/EMnify/matrix-synapse-auto-deploy) Ansible deployment, but:
- this one is a complete Ansible playbook (instead of just a role), so it's **easier to run** - especially for folks not familiar with Ansible
- this one installs and hooks together **a lot more Matrix-related services** for you (see above)
- this one **can be executed more than once** without causing trouble
- works on various distros: **CentOS** (7.0+), Debian-based distributions (**Debian** 10/Buster+, **Ubuntu** 18.04+), **Archlinux**
- this one installs everything in a single directory (`/matrix` by default) and **doesn't "contaminate" your server** with files all over the place
- this one **doesn't necessarily take over** ports 80 and 443. By default, it sets up nginx for you there, but you can also [use your own webserver](configuring-playbook-own-webserver.md)
- this one **runs everything in Docker containers**, so it's likely more predictable and less fragile (see [Docker images used by this playbook](container-images.md))
- this one retrieves and automatically renews free [Let's Encrypt](https://letsencrypt.org/) **SSL certificates** for you
- this one optionally can store the `media_store` content repository files on [Amazon S3](https://aws.amazon.com/s3/) (but defaults to storing files on the server's filesystem)
- this one optionally **allows you to use an external PostgreSQL server** for Synapse's database (but defaults to running one in a container)
- helps you **import data from a previous installation** (so you can migrate your manual virtualenv/Docker setup to a more managed one)
- this one is actually **maintained**
## Server-related
### What kind of server do I need to install Matrix using this Ansible playbook?
We list our server requirements in [Prerequisites](prerequisites.md).
### Why not run Matrix on Kubernetes?
There's no reason not to run Matrix on [Kubernetes](https://kubernetes.io/).
However, that's overly complicated for thousands of us who just want to run a single small (and sometimes not so small) Matrix server, either using "cloud" servers or even a [Raspberry Pi](https://www.raspberrypi.org/) at home.
For us, a Kubernetes-based setup which requires a cluster of multiple computers and is more technically-involved is a no-go.
There are others working on automating a Matrix-on-Kubernetes setup, such as this [Helm](https://helm.sh/) chart: https://github.com/dacruz21/matrix-chart.
### Why don't you use Podman instead of Docker?
We like the philosophy of a daemonless container runtime, but [Podman](https://podman.io) is just not ready for our use case yet.
Learn more about our past experiences/attempts to give Podman a chance, by reading [this issue](https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/520).
In short, `alias podman=docker` is a lie (for us).
### Why use Docker?
[Docker](https://www.docker.com/) is one of our 2 hard dependencies (the other one being [systemd](https://systemd.io/)).
It lets us run services in an isolated manner and independently of the (usually old) packages available for distributions.
It also lets us have a unified setup which runs the same across various supported distros (see them on [Prerequisites](prerequisites.md)).
### Is Docker a hard requirement?
Yes. See [Why don't you use Podman instead of Docker?](#why-dont-you-use-podman-instead-of-docker) for why we're not using another container runtime.
All of our services run in containers. It's how we achieve predictability and also how we support tens of different services across lots of distros.
The only thing we need on the distro is systemd and Python (we install Docker ourselves, unless you ask us not to).
### Why don't you use docker-compose?
Instead of using [docker-compose](https://docs.docker.com/compose/), we prefer installing systemd services and scheduling those independently.
There are people who have worked on turning this setup into a docker-compose-based one. See these experiments [here](https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/64#issuecomment-603164625).
### Can I run this on a distro without systemd?
No. [systemd](https://systemd.io/) is one of our 2 hard dependencies (the other one being [Docker](https://www.docker.com/)).
### Can I install this on a Raspberry Pi?
Yes, you can. See our [Alternative Architectures](alternative-architectures.md) documentation page.
Whether a Raspberry Pi has enough power to give you a good experience is another question. It depends on your use case.
Also see: [What kind of server specs do I need?](#what-kind-of-server-specs-do-i-need).
### What kind of server specs do I need?
This largely depends on your use case. It's not so much the number of users that you plan to host, but rather the number of large rooms they will join.
Federated rooms with lots of history and containing hundreds of other servers are very heavy CPU-wise and memory-wise.
You can probably use a 1 CPU + 1GB memory server to host hundreds of local users just fine, but as soon as one of them joins a federated room like `#matrix:matrix.org` (Matrix HQ) or some IRC-bridged room (say `##linux`), your server will get the need for a lot more power (at least 2GB RAM, etc).
Running Matrix on a server with 1GB of memory is possible (especially if you disable some not-so-important services). See [How do I optimize this setup for a low-power server?](#how-do-i-optimize-this-setup-for-a-low-power-server).
**We recommend starting with a server having at least 2GB of memory** and even then using it sparingly. If you know for sure you'll be joining various large rooms, etc., then going for 4GB of memory or more is a good idea.
Besides the regular Matrix stuff, we also support things like video-conferencing using [Jitsi](configuring-playbook-jitsi.md) and other additional services which (when installed) may use up a lot of memory. Things do add up. Besides the Synapse Matrix server, Jitsi is especially notorious for consuming a lot of resources. If you plan on running Jitsi, we recommend a server with at least 2GB of memory (preferrably more). See our [Jitsi documentation page](configuring-playbook-jitsi.md) to learn how to optimize its memory/CPU usage.
### Can I run this in an LXC container?
If your distro runs within an [LXC container](https://linuxcontainers.org/), you may hit [this issue](https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/703). It can be worked around, if absolutely necessary, but we suggest that you avoid running from within an LXC container.
## Configuration
### Why install my server at matrix.DOMAIN and not at the base DOMAIN?
It's the same with email servers. Your email address is likely `name@company.com`, not `name@mail.company.com`, even though it's `mail.company.com` that is really handling your data for `@company.com` email to work.
Using a separate domain name is easier to manage (although it's a little hard to get right at first) and keeps your Matrix server isolated from your website (if you have one), from your email server (if you have one), etc.
We allow `matrix.DOMAIN` to be the Matrix server handling Matrix stuff for `DOMAIN` by [Server Delegation](howto-server-delegation.md). During the installation procedure, we recommend that you set up server delegation using the [.well-known](configuring-well-known.md) method.
If you'd really like to install Matrix services directly on the base domain, see [How do I install on matrix.DOMAIN without involving the base DOMAIN?](#how-do-i-install-on-matrixdomain-without-involving-the-base-domain).
### I don't control anything on the base domain and can't set up delegation to matrix.DOMAIN. What do I do?
If you're not in control of your base domain (or the server handling it) at all, you can take a look at [How do I install on matrix.DOMAIN without involving the base DOMAIN?](#how-do-i-install-on-matrixdomain-without-involving-the-base-domain)
### I can't set up HTTPS on the base domain. How will I get Matrix federating?
If you really can't obtain an HTTPS certificate for your base domain, you can take a look at [How do I install on matrix.DOMAIN without involving the base DOMAIN?](#how-do-i-install-on-matrixdomain-without-involving-the-base-domain)
### How do I install on matrix.DOMAIN without involving the base DOMAIN?
This Ansible playbook guides you into installing a server for `DOMAIN` (user identifiers are like this: `@user:DOMAIN`), while the server is at `matrix.DOMAIN`.
We allow `matrix.DOMAIN` to be the Matrix server handling Matrix stuff for `DOMAIN` by [Server Delegation](howto-server-delegation.md). During the installation procedure, we recommend that you set up server delegation using the [.well-known](configuring-well-known.md) method.
If you're fine with uglier identifiers (`@user:matrix.DOMAIN`, which is the equivalent of having an email address like `bob@mail.company.com`, instead of just `bob@company.com`), you can do that as well using the following configuration in your `vars.yml` file:
```yaml
# This is what your identifiers are like (e.g. `@bob:matrix.YOUR_BASE_DOMAIN`).
# This is where you access Jitsi (if enabled via `matrix_jitsi_enabled: true`; NOT enabled by default).
#
# Feel free to use `jitsi.matrix.YOUR_BASE_DOMAIN`, if you'd prefer that.
matrix_server_fqn_jitsi: "jitsi.YOUR_BASE_DOMAIN"
```
### I don't use the base domain for anything. How am I supposed to set up Server Delegation for Matrix services?
If you don't use your base domain for anything, then it's hard for you to "serve files over HTTPS" on it -- something we ask you to do for the [.well-known](configuring-well-known.md) setup (needed for [Server Delegation](howto-server-delegation.md)).
Luckily, the playbook can set up your Matrix server (at `matrix.DOMAIN`) to also handle traffic for the base domain (`DOMAIN`).
See [Serving the base domain](configuring-playbook-base-domain-serving.md).
### How do I optimize this setup for a low-power server?
You can disable some not-so-important services to save on memory.
```yaml
# An identity server is not a must.
matrix_ma1sd_enabled: false
# Disabling this will prevent email-notifications and other such things from working.
matrix_mailer_enabled: false
# You can also disable this to save more RAM,
# at the expense of audio/video calls being unreliable.
matrix_coturn_enabled: false
# This makes Synapse not keep track of who is online/offline.
#
# Keeping track of this and announcing such online-status in federated rooms with
# hundreds of servers inside is insanely heavy (https://github.com/matrix-org/synapse/issues/3971).
#
# If your server does not federate with hundreds of others, enabling this doesn't hurt much.
matrix_synapse_presence_enabled: false
```
You can also consider implementing a restriction on room complexity, in order to prevent users from joining very heavy rooms:
```yaml
matrix_synapse_configuration_extension_yaml: |
limit_remote_rooms:
enabled: true
complexity: 1.0 # this limits joining complex (~large) rooms, can be
# increased, but larger values can require more RAM
```
If you've installed [Jitsi](configuring-playbook-jitsi.md) (not installed by default), there are additional optimizations listed on its documentation page that you can perform.
### I already have Docker on my server. Can you stop installing Docker via the playbook?
Yes, we can stop installing Docker ourselves. Just use this in your `vars.yml` file:
```yaml
matrix_docker_installation_enabled: true
```
### I run another webserver on the same server where I wish to install Matrix. What now?
By default, we install a webserver for you (nginx), but you can also use [your own webserver](configuring-playbook-own-webserver.md).
### How is the effective configuration determined?
Configuration variables are defined in multiple places in this playbook and are considered in this order:
- there are defaults coming from each role's defaults file (`role/matrix*/defaults/main.yml`). These variable values aim to be good defaults for when the role is used standalone (outside of this collection of roles, also called playbook).
- then, there are overrides in `group_vars/matrix_servers`, which aim to adjust these "standalone role defaults" to something which better fits the playbook in its entirety.
- finally, there's your `inventory/host_vars/matrix.DOMAIN/vars.yml` file, which is the ultimate override
### What configuration variables are available?
You can discover the variables you can override in each role (`role/matrix*/defaults/main.yml`).
As described in [How is the effective configuration determined?](#how-is-the-effective-configuration-determined), these role-defaults may be overriden by values defined in `group_vars/matrix_servers`.
Refer to both of these for inspiration. Still, as mentioned in [Configuring the playbook](configuring-playbook.md), you're only ever supposed to edit your own `inventory/host_vars/matrix.DOMAIN/vars.yml` file and nothing else inside the playbook (unless you're meaning to contribute new features).
### I'd like to adjust some configuration which doesn't have a corresponding variable. How do I do it?
The playbook doesn't aim to expose all configuration settings for all services using variables.
Doing so would amount to hundreds of variables that we have to create and maintain.
Instead, we only try to make some important basics configurable using dedicated variables you can see in each role.
See [What configuration variables are available?](#what-configuration-variables-are-available).
Besides that, each role (component) aims to provide a `matrix_SOME_COMPONENT_configuration_extension_yaml` (or `matrix_SOME_COMPONENT_configuration_extension_json`) variable, which can be used to override the configuration.
Check each role's `role/matrix*/defaults/main.yml` for the corresponding variable and an example for how use it.
## Installation
### How do I run the installation?
See [Installing](installing.md) to learn how to use Ansible to install Matrix services.
Of course, don't just jump straight to Installing. Rather, start at [Prerequisites](prerequisites.md) and get guided from there (into [setting up DNS](configuring-dns.md), [configuring the playbook](configuring-playbook.md), etc).
### I installed Synapse some other way. Can I migrate such a setup to the playbook?
Yes, you can.
You generally need to do a playbook installation (start at the [Prerequisites](prerequisites.md) page), followed by importing your existing data into it.
This Ansible playbook guides you into installing a server for `DOMAIN` (user identifiers are like this: `@user:DOMAIN`), while the server is at `matrix.DOMAIN`. If your existing setup has a server name (`server_name` configuration setting in Synapse's `homeserver.yaml` file) other than the base `DOMAIN`, you may need to tweak some additional variables. This FAQ entry may be of use if you're dealing with a more complicated setup - [How do I install on matrix.DOMAIN without involving the base DOMAIN?](#how-do-i-install-on-matrixdomain-without-involving-the-base-domain)
After configuring the playbook and installing and **before starting** services (done with `ansible-playbook ... --tags=start`) you'd import [your SQLite](importing-synapse-sqlite.md) (or [Postgres](importing-postgres.md)) database and also [import your media store](importing-synapse-media-store.md).
### I've downloaded Ansible and the playbook on the server. It can't connect using SSH.
If you're using the playbook directly on the server, then Ansible doesn't need to connect using SSH.
It can perform a local connection instead. Just set `ansible_connection=local` at the end of the server line in `inventory/hosts` and re-run the playbook.
If you're running Ansible from within a container (one of the possibilities we list on our [dedicated Ansible documentation page](ansible.md)), then using `ansible_connection=local` is not possible.
## Troubleshooting
### I get "Error response from daemon: configured logging driver does not support reading" when I do `docker logs matrix-synapse`.
See [How can I see the logs?](#how-can-i-see-the-logs).
### How can I see the logs?
We utilize [systemd/journald](https://www.freedesktop.org/software/systemd/man/systemd-journald.service.html#Description) for logging.
To see logs for Synapse, run `journalctl -fu matrix-synapse.service`. You may wish to see the [manual page for journalctl](https://www.commandlinux.com/man-page/man1/journalctl.1.html).
Available service names can be seen by doing `ls /etc/systemd/system/matrix*.service` on the server.
Some services also log to files in `/matrix/*/data/..`, but we're slowly moving away from that.
We also disable Docker logging, so you can't use `docker logs matrix-*` either. We do this to prevent useless double (or even triple) logging and to avoid having to rotate log files.
We just simply delegate logging to journald and it takes care of persistence and expiring old data.
Also see: [How long do systemd/journald logs persist for?](#how-long-do-systemdjournald-logs-persist-for)
### How long do systemd/journald logs persist for?
On some distros, the journald logs are just in-memory and not persisted to disk.
Consult (and feel free to adjust) your distro's journald logging configuration in `/etc/systemd/journald.conf`.
To enable persistence and put some limits on how large the journal log files can become, adjust your configuration like this:
```ini
[Journal]
RuntimeMaxUse=200M
SystemMaxUse=1G
RateLimitInterval=0
RateLimitBurst=0
Storage=persistent
```
## Maintenance
### Do I need to do anything to keep my Matrix server updated?
Yes. We don't update anything for you automatically.
See our [documentation page about upgrading services](maintenance-upgrading-services.md).
### How do I move my existing installation to another (VM) server?
If you have an existing installation done using this Ansible playbook, you can easily migrate that to another server using [our dedicated server migration guide](maintenance-migrating.md).
If your previous installation is done in some other way (not using this Ansible playbook), see [I installed Synapse some other way. Can I migrate such a setup to the playbook?](#i-installed-synapse-some-other-way-can-i-migrate-such-a-setup-to-the-playbook).
### How do I back up the data on my server?
We haven't documented this properly yet, but the general advice is to:
- back up Postgres by making a database dump. See [Backing up PostgreSQL](maintenance-postgres.md#backing-up-postgresql)
- back up all `/matrix` files, except for `/matrix/postgres/data` (you already have a dump) and `/matrix/postgres/data-auto-upgrade-backup` (this directory may exist and contain your old data if you've [performed a major Postgres upgrade](maintenance-postgres.md#upgrading-postgresql)).
You can later restore these roughly like this:
- restore the `/matrix` directory and files on the new server manually
- run the playbook again (see [Installing](installing.md)), but **don't** start services yet (**don't run** `... --tags=start`). This step will fix any file permission mismatches and will also set up additional software (Docker, etc.) and files on the server (systemd service, etc.).
- perform a Postgres database import (see [Importing Postgres](importing-postgres.md)) to restore your database backup
- start services (see [Starting the services](installing.md#starting-the-services))
If your server's IP address has changed, you may need to [set up DNS](configuring-dns.md) again.
### What is this `/matrix/postgres/data-auto-upgrade-backup` directory that is taking up so much space?
When you [perform a major Postgres upgrade](maintenance-postgres.md#upgrading-postgresql), we save the the old data files in `/matrix/postgres/data-auto-upgrade-backup`, just so you could easily restore them should something have gone wrong.
After verifying that everything still works after the Postgres upgrade, you can safely delete `/matrix/postgres/data-auto-upgrade-backup`
### How do I debug or force SSL certificate renewal?
SSL certificate renewal normally happens automatically via [systemd timers](https://wiki.archlinux.org/index.php/Systemd/Timers).
If you're having trouble with SSL certificate renewal, you can inspect the renewal logs using:
- *or* by looking at the log files in `/matrix/ssl/log/`
To trigger renewal, run: `systemctl start matrix-ssl-lets-encrypt-certificates-renew.service`. You can then take a look at the logs again.
If you're using the integrated webserver (`matrix-nginx-proxy`), you can reload it manually like this: `systemctl reload matrix-nginx-proxy`. Reloading also happens periodically via a systemd timer.
If you're [using your own webserver](configuring-playbook-own-webserver.md) instead of the integrated one (`matrix-nginx-proxy`) you may also need to reload/restart it, to make it pick up the renewed SSL certificate files.
This Ansible playbook is meant to be executed on your own computer (not the Matrix server).
In special cases (if your computer cannot run Ansible, etc.) you may put the playbook on the server as well.
You can retrieve the playbook's source code by:
- [Using git to get the playbook](#using-git-to-get-the-playbook) (recommended)
- [Downloading the playbook as a ZIP archive](#downloading-the-playbook-as-a-zip-archive) (not recommended)
## Using git to get the playbook
We recommend using the [git](https://git-scm.com/) tool to get the playbook's source code, because it lets you easily keep up to date in the future when [Maintaining services](maintenance-upgrading-services.md).
Once you've installed git on your computer, you can go to any directory of your choosing and run the following command to retrieve the playbook's source code:
To have a server on a subdomain (e.g. `matrix.<your-domain>`) handle Matrix federation traffic for the base domain (`<your-domain>`), we need to instruct the Matrix network of such a delegation.
By default, this playbook guides you into setting up [Server Delegation via a well-known file](#server-delegation-via-a-well-known-file).
However, that method may have some downsides that are not to your liking. Hence this guide about alternative ways to set up Server Delegation.
It is a complicated matter, so unless you are affected by the [Downsides of well-known-based Server Delegation](#downsides-of-well-known-based-server-delegation), we suggest you stay on the simple/default path.
## Server Delegation via a well-known file
Serving a `/.well-known/matrix/server` file from the base domain is the most straightforward way to set up server delegation, but it suffers from some problems that we list in [Downsides of well-known-based Server Delegation](#downsides-of-well-known-based-server-delegation).
As we already mention in [Configuring DNS](configuring-dns.md) and [Configuring Service Discovery via .well-known](configuring-well-known.md),
this playbook already properly guides you into setting up such delegation by means of a `/.well-known/matrix/server` file served from the base domain (`<your-domain>`).
If this is okay with you, feel free to not read ahead.
### Downsides of well-known-based Server Delegation
Server Delegation by means of a `/.well-known/matrix/server` file is the most straightforward, but suffers from the following downsides:
- you need to have a working HTTPS server for the base domain (`<your-domain>`). If you don't have any server for the base domain at all, you can easily solve it by making the playbook [serve the base domain from the Matrix server](configuring-playbook-base-domain-serving.md).
- any downtime on the base domain (`<your-domain>`) or network trouble between the matrix subdomain (`matrix.<your-domain>`) and the base `<domain>` may cause Matrix Federation outages. As the [Server-Server spec says](https://matrix.org/docs/spec/server_server/r0.1.0.html#server-discovery):
> Errors are recommended to be cached for up to an hour, and servers are encouraged to exponentially back off for repeated failures.
**For most people, this is a reasonable tradeoff** given that it's easy and straightforward to set up. We recommend you stay on this path.
Otherwise, you can decide to go against the default for this playbook, and instead set up [Server Delegation via a DNS SRV record (advanced)](#server-delegation-via-a-dns-srv-record-advanced) (much more complicated).
## Server Delegation via a DNS SRV record (advanced)
**NOTE**: doing Server Delegation via a DNS SRV record is a more **advanced** way to do it and is not the default for this playbook. This is usually **much more complicated** to set up, so **we don't recommend it**. If you're not an experience sysadmin, you'd better stay away from this.
As per the [Server-Server spec](https://matrix.org/docs/spec/server_server/r0.1.0.html#server-discovery), it's possible to do Server Delegation using only a SRV record (without a `/.well-known/matrix/server` file).
This prevents you from suffering the [Downsides of well-known-based Server Delegation](#downsides-of-well-known-based-server-delegation).
To use DNS SRV record validation, you need to:
- ensure that `/.well-known/matrix/server` is **not served** from the base domain, as that would interfere with DNS SRV record Server Delegation. To make the playbook **not** generate and serve the file, use the following configuration: `matrix_well_known_matrix_server_enabled: false`.
- ensure that you have a `_matrix._tcp` DNS SRV record for your base domain (`<your-domain>`) with a value of `10 0 8448 matrix.<your-domain>`
- ensure that you are serving the Matrix Federation API (tcp/8448) with a certificate for `<your-domain>` (not `matrix.<your-domain>`!). Getting this certificate to the `matrix.<your-domain>` server may be complicated. The playbook's automatic SSL obtaining/renewal flow will likely not work and you'll need to copy certificates around manually. See below.
### Obtaining certificates
How you can obtain a valid certificate for `<your-domain>` on the `matrix.<your-domain>` server is up to you.
If `<your-domain>` and `matrix.<your-domain>` are hosted on the same machine, you can let the playbook obtain the certificate for you, by following our [Obtaining SSL certificates for additional domains](configuring-playbook-ssl-certificates.md#obtaining-ssl-certificates-for-additional-domains) guide.
If `<your-domain>` and `matrix.<your-domain>` are not hosted on the same machine, you can copy over the certificate files manually.
Don't forget that they may get renewed once in a while, so you may also have to transfer them periodically. How often you do that is up to you, as long as the certificate files don't expire.
### Serving the Federation API with your certificates
Regardless of which method for obtaining certificates you've used, once you've managed to get certificates for your base domain onto the `matrix.<your-domain>` machine you can put them to use.
Based on your setup, you have different ways to go about it:
- [Serving the Federation API with your certificates and matrix-nginx-proxy](#serving-the-federation-api-with-your-certificates-and-matrix-nginx-proxy)
- [Serving the Federation API with your certificates and another webserver](#serving-the-federation-api-with-your-certificates-and-another-webserver)
- [Serving the Federation API with your certificates and Synapse handling Federation](#serving-the-federation-api-with-your-certificates-and-synapse-handling-federation)
### Serving the Federation API with your certificates and matrix-nginx-proxy
**If you are using matrix-nginx-proxy**, a reverse-proxy webserver used by default in this playbook, you only need to override the certificates used for the Matrix Federation API. You can do that using:
```yaml
# Adjust paths below to point to your certificate.
#
# NOTE: these are in-container paths. `/matrix/ssl` on the host is mounted into the container
# at the same path (`/matrix/ssl`) by default, so if that's the path you need, it would be seamless.
You then refer to them (for `matrix_nginx_proxy_proxy_matrix_federation_api_ssl_certificate` and `matrix_nginx_proxy_proxy_matrix_federation_api_ssl_certificate_key`) by using `/some/path/inside/the/container`.
Make sure to reload matrix-nginx-proxy once in a while (`systemctl reload matrix-nginx-proxy`), so that newer certificates can kick in.
Reloading doesn't cause any downtime.
### Serving the Federation API with your certificates and another webserver
**If you are NOT using matrix-nginx-proxy**, but rather some other webserver, you can set up reverse-proxying for the `tcp/8448` port by yourself.
Make sure to use the proper certificates for `<your-domain>` (not for `matrix.<your-domain>`) when serving the `tcp/8448` port.
Proxying needs to happen to `127.0.0.1:8048` (unencrypted Synapse federation listener).
Make sure to reload/restart your webserver once in a while, so that newer certificates can kick in.
### Serving the Federation API with your certificates and Synapse handling Federation
**Alternatively**, if you are **NOT using matrix-nginx-proxy** and **would rather not use your own webserver for Federation traffic**, you can let Synapse handle Federation by itself.
To do that, make sure the certificate files are mounted into the Synapse container:
**Note**: `<server-path-to-postgres-dump.sql>` must be a file path to a Postgres dump file on the server (not on your local machine!).
## Troubleshooting
A table ownership issue can occur if you are importing from a Synapse installation which was both:
- migrated from SQLite to Postgres, and
- used a username other than 'synapse'
In this case you may run into the following error during the import task:
```
"ERROR: role \"synapse_user\" does not exist"
```
where `synapse_user` is the database username from the previous Synapse installation.
This can be verified by examining the dump for ALTER TABLE statements which set OWNER TO that username:
```Shell
$ grep "ALTER TABLE" homeserver.sql"
ALTER TABLE public.access_tokens OWNER TO synapse_user;
ALTER TABLE public.account_data OWNER TO synapse_user;
ALTER TABLE public.account_data_max_stream_id OWNER TO synapse_user;
ALTER TABLE public.account_validity OWNER TO synapse_user;
ALTER TABLE public.application_services_state OWNER TO synapse_user;
...
```
It can be worked around by changing the username to `synapse`, for example by using `sed`:
```Shell
$ sed -i "s/synapse_user/synapse/g" homeserver.sql
```
This uses sed to perform an 'in-place' (`-i`) replacement globally (`/g`), searching for `synapse user` and replacing with `synapse` (`s/synapse_user/synapse`). If your database username was different, change `synapse_user` to that username instead.
Note that if the previous import failed with an error it may have made changes which are incompatible with re-running the import task right away; if you do so it may fail with an error such as:
```
ERROR: relation \"access_tokens\" already exists
```
In this case you can use the command suggested in the import task to clear the database before retrying the import:
```Shell
# systemctl stop matrix-postgres
# rm -rf /matrix/postgres/data/*
# systemctl start matrix-postgres
```
Once the database is clear and the ownership of the tables has been fixed in the SQL file, the import task should succeed.
# Importing `media_store` data files from an existing Synapse installation (optional)
Run this if you'd like to import your `media_store` files from a previous installation of Synapse.
## Prerequisites
Before doing the actual data restore, **you need to upload your media store directory to the server** (any path is okay).
If you are [Storing Matrix media files on Amazon S3](configuring-playbook-s3.md) (optional), restoring with this tool is not possible right now.
As an alternative, you can perform a manual restore using the [AWS CLI tool](https://aws.amazon.com/cli/) (e.g. `aws s3 sync /path/to/server/media_store/. s3://name-of-bucket/`)
**Note for Mac users**: Due to case-sensitivity issues on certain Mac filesystems (HFS or HFS+), filename corruption may occur if you copy a `media_store` directory to your Mac. If you're transferring a `media_store` directory between 2 servers, make sure you do it directly (from server to server with a tool such as [rsync](https://rsync.samba.org/)), and not by downloading the files to your Mac.
## Importing
Run this command (make sure to replace `<server-path-to-media_store>` with a path on your server):
- `<server-path-to-homeserver.db>` must be a file path to a `homeserver.db`**file on the server** (not on your local machine!).
- if the SQLite database is from an older version of Synapse, the **importing procedure may run migrations on it to bring it up to date**. That is, your SQLite database file may get modified and become unusable with your older Synapse version. Keeping a copy of the original is probably wise.
If you've [configured your DNS](configuring-dns.md) and have [configured the playbook](configuring-playbook.md), you can start the installation procedure.
**Note**: if you don't use SSH keys for authentication, but rather a regular password, you may need to add `--ask-pass` to the above (and all other) Ansible commands.
**Note**: if you **do** use SSH keys for authentication, **and** use a non-root user to *become* root (sudo), you may need to add `-K` (`--ask-become-pass`) to the above (and all other) Ansible commands.
The above command **doesn't start any services just yet** (another step does this later - below).
Feel free to **re-run this setup command any time** you think something is off with the server configuration.
## Things you might want to do after installing
After installing, but before starting the services, you may want to do additional things like:
- [Importing an existing SQLite database (from another Synapse installation)](importing-synapse-sqlite.md) (optional)
- [Importing an existing Postgres database (from another installation)](importing-postgres.md) (optional)
- [Importing `media_store` data files from an existing Synapse installation](importing-synapse-media-store.md) (optional)
## Starting the services
When you're ready to start the Matrix services (and set them up to auto-start in the future):
Now that services are running, you need to **finalize the installation process** (required for federation to work!) by [Configuring Service Discovery via .well-known](configuring-well-known.md)
## Things to do next
If you have started services and **finalized the installation process** (required for federation to work!) by [Configuring Service Discovery via .well-known](configuring-well-known.md), you can:
- [check if services work](maintenance-checking-services.md)
- or [create your first Matrix user account](registering-users.md)
- or [set up additional services](configuring-playbook.md#other-configuration-options) (bridges to other chat networks, bots, etc.)
- or learn how to [upgrade services when new versions are released](maintenance-upgrading-services.md)
- or learn how to [maintain your server](faq.md#maintenance)
- or join some Matrix rooms:
* via the *Explore rooms* feature in Element or some other client, or by discovering them using this [matrix-static list](https://view.matrix.org). Note: joining large rooms may overload small servers.
* or come say Hi in our support room - [#matrix-docker-ansible-deploy:devture.com](https://matrix.to/#/#matrix-docker-ansible-deploy:devture.com). You might learn something or get to help someone else new to Matrix hosting.
- or help make this playbook better by contributing (code, documentation, or [coffee/beer](https://liberapay.com/s.pantaleev/donate))
Active: active (running) since Wed 2018-11-14 19:38:35 UTC; 49min ago
```
You can see the logs by using journalctl. Example:
```
sudo journalctl -fu matrix-synapse
```
## Increasing Synapse logging
Because the [Synapse](https://github.com/matrix-org/synapse) Matrix server is originally very chatty when it comes to logging, we intentionally reduce its [logging level](https://docs.python.org/3/library/logging.html#logging-levels) from `INFO` to `WARNING`.
If you'd like to debug an issue or [report a Synapse bug](https://github.com/matrix-org/synapse/issues/new/choose) to the developers, it'd be better if you temporarily increasing the logging level to `INFO`.
Example configuration (`inventory/host_vars/matrix.DOMAIN/vars.yml`):
```yaml
matrix_synapse_log_level: "INFO"
matrix_synapse_storage_sql_log_level: "INFO"
matrix_synapse_root_log_level: "INFO"
```
Re-run the playbook after making these configuration changes.
## Remove unused Docker data
You can free some disk space from Docker, see [docker system prune](https://docs.docker.com/engine/reference/commandline/system_prune/) for more information.
> **Note**: This migration guide is applicable if you migrate from one server to another server having the same CPU architecture (e.g. both servers being `amd64`).
>
> If you're trying to migrate between different architectures (e.g. `amd64` --> `arm64`), simply copying the complete `/matrix` directory is not possible as it would move the raw PostgreSQL data between different architectures. In this specific case, you can use the guide below as a reference, but you would also need to dump the database on your current server and import it properly on the new server. See our [Backing up PostgreSQL](maintenance-postgres.md#backing-up-postgresql) docs for help with PostgreSQL backup/restore.
# Migrating to new server
1. Prepare by lowering DNS TTL for your domains (`matrix.DOMAIN`, etc.), so that DNS record changes (step 4 below) would happen faster, leading to less downtime
2. Stop all services on the old server and make sure they won't be starting again. Execute this on the old server: `systemctl disable --now matrix*`
3. Copy directory `/matrix` from the old server to the new server. Make sure to preserve ownership and permissions (use `cp -p` or `rsync -ar`)!
4. Make sure your DNS records are adjusted to point to the new server's IP address
5. Remove old server from the `inventory/hosts` file and add new server.
6. Run `ansible-playbook -i inventory/hosts setup.yml --tags=setup-system-user`. This will create the `matrix` user and group on the new server
7. Because the `matrix` user and group are created dynamically on each server, the user/group id may differ between the old and new server. We suggest that you adjust ownership of `/matrix` files manually by running this on the new server: `chown -R matrix:matrix /matrix`.
8. Run `ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,start` to finish the installation and start all services
This document shows you how to perform various maintenance tasks related to the Postgres database server used by Matrix.
Table of contents:
- [Getting a database terminal](#getting-a-database-terminal), for when you wish to execute SQL queries
- [Vacuuming PostgreSQL](#vacuuming-postgresql), for when you wish to run a Postgres [VACUUM](https://www.postgresql.org/docs/current/sql-vacuum.html) (optimizing disk space)
- [Backing up PostgreSQL](#backing-up-postgresql), for when you wish to make a backup
- [Upgrading PostgreSQL](#upgrading-postgresql), for upgrading to new major versions of PostgreSQL. Such **manual upgrades are sometimes required**.
- [Tuning PostgreSQL](#tuning-postgresql) to make it run faster
## Getting a database terminal
You can use the `/usr/local/bin/matrix-postgres-cli` tool to get interactive terminal access ([psql](https://www.postgresql.org/docs/11/app-psql.html)) to the PostgreSQL server.
If you are using an [external Postgres server](configuring-playbook-external-postgres.md), the above tool will not be available.
By default, this tool puts you in the `matrix` database, which contains nothing.
To see the available databases, run `\list` (or just `\l`).
To change to another database (for example `synapse`), run `\connect synapse` (or just `\c synapse`).
You can then proceed to write queries. Example: `SELECT COUNT(*) FROM users;`
**Be careful**. Modifying the database directly (especially as services are running) is dangerous and may lead to irreversible database corruption.
When in doubt, consider [making a backup](#backing-up-postgresql).
## Vacuuming PostgreSQL
Deleting lots data from Postgres does not make it release disk space, until you perform a `VACUUM` operation.
To perform a `FULL` Postgres [VACUUM](https://www.postgresql.org/docs/current/sql-vacuum.html), run the playbook with `--tags=run-postgres-vacuum`.
**Note**: this will automatically stop Synapse temporarily and restart it later. You'll also need plenty of available disk space in your Postgres data directory (usually `/matrix/postgres/data`).
## Backing up PostgreSQL
To automatically make Postgres database backups on a fixed schedule, see [Setting up postgres backup](configuring-playbook-postgres-backup.md).
To make a one off back up of the current PostgreSQL database, make sure it's running and then execute a command like this on the server:
```bash
/usr/bin/docker exec \
--env-file=/matrix/postgres/env-postgres-psql \
matrix-postgres \
/usr/local/bin/pg_dumpall -h matrix-postgres \
| gzip -c \
> /matrix/postgres.sql.gz
```
If you are using an [external Postgres server](configuring-playbook-external-postgres.md), the above command will not work, because neither the credentials file (`/matrix/postgres/env-postgres-psql`), nor the `matrix-postgres` container is available.
Restoring a backup made this way can be done by [importing it](importing-postgres.md).
## Upgrading PostgreSQL
Unless you are using an [external Postgres server](configuring-playbook-external-postgres.md), this playbook initially installs Postgres for you.
Once installed, the playbook attempts to preserve the Postgres version it starts with.
This is because newer Postgres versions cannot start with data generated by older Postgres versions.
Upgrades must be performed manually.
This playbook can upgrade your existing Postgres setup with the following command:
**The old Postgres data directory is backed up** automatically, by renaming it to `/matrix/postgres/data-auto-upgrade-backup`.
To rename to a different path, pass some extra flags to the command above, like this: `--extra-vars="postgres_auto_upgrade_backup_data_path=/another/disk/matrix-postgres-before-upgrade"`
The auto-upgrade-backup directory stays around forever, until you **manually decide to delete it**.
As part of the upgrade, the database is dumped to `/tmp`, an upgraded and empty Postgres server is started, and then the dump is restored into the new server.
To use a different directory for the dump, pass some extra flags to the command above, like this: `--extra-vars="postgres_dump_dir=/directory/to/dump/here"`
To save disk space in `/tmp`, the dump file is gzipped on the fly at the expense of CPU usage.
If you have plenty of space in `/tmp` and would rather avoid gzipping, you can explicitly pass a dump filename which doesn't end in `.gz`.
**All databases, roles, etc. on the Postgres server are migrated**.
## Tuning PostgreSQL
PostgreSQL can be tuned to make it run faster. This is done by passing extra arguments to Postgres with the `matrix_postgres_process_extra_arguments` variable. You should use a website like https://pgtune.leopard.in.ua/ or information from https://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server to determine what Postgres settings you should change.
**Note**: the configuration generator at https://pgtune.leopard.in.ua/ adds spaces around the `=` sign, which is invalid. You'll need to remove it manually (`max_connections = 300` -> `max_connections=300`)
### Here are some examples:
These are not recommended values and they may not work well for you. This is just to give you an idea of some of the options that can be set. If you are an experienced PostgreSQL admin feel free to update this documentation with better examples.
Here is an example config for a small 2 core server with 4GB of RAM and SSD storage:
```
matrix_postgres_process_extra_arguments: [
"-c shared_buffers=128MB",
"-c effective_cache_size=2304MB",
"-c effective_io_concurrency=100",
"-c random_page_cost=2.0",
"-c min_wal_size=500MB",
]
```
Here is an example config for a 4 core server with 8GB of RAM on a Virtual Private Server (VPS); the paramters have been configured using https://pgtune.leopard.in.ua with the following setup: PostgreSQL version 12, OS Type: Linux, DB Type: Mixed type of application, Data Storage: SSD storage:
```
matrix_postgres_process_extra_arguments: [
"-c max_connections=100",
"-c shared_buffers=2GB",
"-c effective_cache_size=6GB",
"-c maintenance_work_mem=512MB",
"-c checkpoint_completion_target=0.9",
"-c wal_buffers=16MB",
"-c default_statistics_target=100",
"-c random_page_cost=1.1",
"-c effective_io_concurrency=200",
"-c work_mem=5242kB",
"-c min_wal_size=1GB",
"-c max_wal_size=4GB",
"-c max_worker_processes=4",
"-c max_parallel_workers_per_gather=2",
"-c max_parallel_workers=4",
"-c max_parallel_maintenance_workers=2",
]
```
Here is an example config for a large 6 core server with 24GB of RAM:
This document shows you how to perform various maintenance tasks related to the Synapse chat server.
Table of contents:
- [Purging old data with the Purge History API](#purging-old-data-with-the-purge-history-api), for when you wish to delete in-use (but old) data from the Synapse database
- [Compressing state with rust-synapse-compress-state](#compressing-state-with-rust-synapse-compress-state)
- [Browse and manipulate the database](#browse-and-manipulate-the-database), for when you really need to take matters into your own hands
- [Make Synapse faster](#make-synapse-faster)
## Purging old data with the Purge History API
You can use the **[Purge History API](https://github.com/matrix-org/synapse/blob/master/docs/admin_api/purge_history_api.md)** to delete old messages on a per-room basis. **This is destructive** (especially for non-federated rooms), because it means **people will no longer have access to history past a certain point**.
To make use of this API, **you'll need an admin access token** first. You can find your access token in the setting of some clients (like Element).
Alternatively, you can log in and obtain a new access token like this:
Synapse's Admin API is not exposed to the internet by default. To expose it you will need to add `matrix_nginx_proxy_proxy_matrix_client_api_forwarded_location_synapse_admin_api_enabled: true` to your `vars.yml` file.
Follow the [Purge History API](https://github.com/matrix-org/synapse/blob/master/docs/admin_api/purge_history_api.md) documentation page for the actual purging instructions.
After deleting data, you may wish to run a [`FULL` Postgres `VACUUM`](./maintenance-postgres.md#vacuuming-postgresql).
## Compressing state with rust-synapse-compress-state
[rust-synapse-compress-state](https://github.com/matrix-org/rust-synapse-compress-state) can be used to optimize some `_state` tables used by Synapse. If your server participates in large rooms this is the most effective way to reduce the size of your database.
This tool should be safe to use (even when Synapse is running), but it's always a good idea to [make Postgres backups](./maintenance-postgres.md#backing-up-postgresql) first.
To ask the playbook to run rust-synapse-compress-state, execute:
By default, all rooms with more than `100000` state group rows will be compressed.
If you need to adjust this, pass: `--extra-vars='matrix_synapse_rust_synapse_compress_state_min_state_groups_required=SOME_NUMBER_HERE'` to the command above.
After state compression, you may wish to run a [`FULL` Postgres `VACUUM`](./maintenance-postgres.md#vacuuming-postgresql).
## Browse and manipulate the database
When the [Synapse Admin API](https://github.com/matrix-org/synapse/tree/master/docs/admin_api) and the other tools do not provide a more convenient way, having a look at synapse's postgresql database can satisfy a lot of admins' needs.
Editing the database manually is not recommended or supported by the Synapse developers. If you are going to do so you should [make a database backup](./maintenance-postgres.md#backing-up-postgresql).
First, set up an SSH tunnel to your matrix server (skip if it is your local machine):
```
# you may replace 1799 with an arbitrary port unbound on both machines
ssh -L 1799:localhost:1799 matrix.DOMAIN
```
Then start up an ephemeral [adminer](https://www.adminer.org/) container on the Matrix server, connecting it to the `matrix` network and linking the postgresql container:
```
docker run --rm --publish 1799:8080 --link matrix-postgres --net matrix adminer
```
You should then be able to browse the adminer database administration GUI at http://localhost:1799/ after entering your DB credentials (found in the `host_vars` or on the server in `{{matrix_synapse_config_dir_path}}/homeserver.yaml` under `database.args`)
⚠️ Be **very careful** with this, there is **no undo** for impromptu DB operations.
## Make Synapse faster
Synapse's presence feature which tracks which users are online and which are offline can use a lot of processing power. You can disable presence by adding `matrix_synapse_presence_enabled: false` to your `vars.yml` file.
Tuning Synapse's cache factor can help reduce RAM usage. [See the upstream documentation](https://github.com/matrix-org/synapse#help-synapse-is-slow-and-eats-all-my-ram-cpu) for more information on what value to set the cache factor to. Use the variable `matrix_synapse_caches_global_factor` to set the cache factor.
Tuning your PostgreSQL database will also make Synapse run significantly faster. See [maintenance-postgres.md##tuning-postgresql](maintenance-postgres.md##tuning-postgresql).
See also [How do I optimize this setup for a low-power server?](faq.md#how-do-i-optimize-this-setup-for-a-low-power-server).
This playbook not only installs the various Matrix services for you, but can also upgrade them as new versions are made available.
If you want to be notified when new versions of Synapse are released, you should join the Synapse Homeowners room: [#homeowners:matrix.org](https://matrix.to/#/#homeowners:matrix.org).
To upgrade services:
- update your playbook directory (`git pull`), so you'd obtain everything new we've done
- take a look at [the changelog](../CHANGELOG.md) to see if there have been any backward-incompatible changes that you need to take care of
- re-run the [playbook setup](installing.md): `ansible-playbook -i inventory/hosts setup.yml --tags=setup-all`
- restart the services: `ansible-playbook -i inventory/hosts setup.yml --tags=start`
**Note**: major version upgrades to the internal PostgreSQL database are not done automatically. To upgrade it, refer to the [upgrading PostgreSQL guide](maintenance-postgres.md#upgrading-postgresql).
To install Matrix services using this Ansible playbook, you need:
- (Recommended) An **x86** server ([What kind of server specs do I need?](faq.md#what-kind-of-server-specs-do-i-need)) running one of these operating systems:
- **CentOS** (7 only for now; [8 is not yet supported](https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/300))
- **Debian** (10/Buster or newer)
- **Ubuntu** (18.04 or newer, although [20.04 may be problematic](ansible.md#supported-ansible-versions))
- **Archlinux**
Generally, newer is better. We only strive to support released stable versions of distributions, not betas or pre-releases. This playbook can take over your whole server or co-exist with other services that you have there.
This playbook somewhat supports running on non-`amd64` architectures like ARM. See [Alternative Architectures](alternative-architectures.md).
If your distro runs within an [LXC container](https://linuxcontainers.org/), you may hit [this issue](https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/703). It can be worked around, if absolutely necessary, but we suggest that you avoid running from within an LXC container.
- `root` access to your server (or a user capable of elevating to `root` via `sudo`).
- [Python](https://www.python.org/) being installed on the server. Most distributions install Python by default, but some don't (e.g. Ubuntu 18.04) and require manual installation (something like `apt-get install python3`). On some distros, Ansible may incorrectly [detect the Python version](https://docs.ansible.com/ansible/latest/reference_appendices/interpreter_discovery.html) (2 vs 3) and you may need to explicitly specify the interpreter path in `inventory/hosts` during installation (e.g. `ansible_python_interpreter=/usr/bin/python3`)
- The [Ansible](http://ansible.com/) program being installed on your own computer. It's used to run this playbook and configures your server for you. Take a look at [our guide about Ansible](ansible.md) for more information, as well as [version requirements](ansible.md#supported-ansible-versions) and alternative ways to run Ansible.
- An HTTPS-capable web server at the base domain name (`<your-domain>`) which is capable of serving static files. Unless you decide to [Serve the base domain from the Matrix server](configuring-playbook-base-domain-serving.md) or alternatively, to use DNS SRV records for [Server Delegation](howto-server-delegation.md).
- Properly configured DNS records for `<your-domain>` (details in [Configuring DNS](configuring-dns.md)).
- Some TCP/UDP ports open. This playbook configures the server's internal firewall for you. In most cases, you don't need to do anything special. But **if your server is running behind another firewall**, you'd need to open these ports:
- `80/tcp`: HTTP webserver
- `443/tcp`: HTTPS webserver
- `3478/tcp`: TURN over TCP (used by Coturn)
- `3478/udp`: TURN over UDP (used by Coturn)
- `5349/tcp`: TURN over TCP (used by Coturn)
- `5349/udp`: TURN over UDP (used by Coturn)
- `8448/tcp`: Matrix Federation API HTTPS webserver. In some cases, this **may necessary even with federation disabled**. Integration Servers (like Dimension) and Identity Servers (like ma1sd) may need to access `openid` APIs on the federation port.
- the range `49152-49172/udp`: TURN over UDP
- potentially some other ports, depending on the additional (non-default) services that you enable in the **configuring the playbook** step (later on). Consult each service's documentation page in `docs/` for that.
When ready to proceed, continue with [Configuring DNS](configuring-dns.md).
**Note**: `<your-username>` is just a plain username (like `john`), not your full `@<username>:<your-domain>` identifier.
**You can then log in with that user** via the Element service that this playbook has created for you at a URL like this: `https://element.<domain>/`.
-----
If you've just installed Matrix, **to finalize the installation process**, it's best if you proceed to [Configuring service discovery via .well-known](configuring-well-known.md)
## Managing users via a Web UI
To manage users more easily (via a web user-interace), you can install [Synapse Admin](configuring-playbook-synapse-admin.md).
## Letting certain users register on your private server
If you'd rather **keep your server private** (public registration closed, as is the default), and **let certain people create accounts by themselves** (instead of creating user accounts manually like this), consider installing and making use of [matrix-registration](configuring-playbook-matrix-registration.md).
## Enabling public user registration
To **open up user registration publicly** (usually **not recommended**), consider using the following configuration:
```yaml
matrix_synapse_enable_registration: true
```
and running the [installation](installing.md) procedure once again.
If you're opening up registrations publicly like this, you might also wish to [configure CAPTCHA protection](configuring-captcha.md).
## Adding/Removing Administrator privileges to an existing user
The script `/usr/local/bin/matrix-change-user-admin-status` may be used to change a user's admin privileges.
* log on to your server with ssh
* execute with the username and 0/1 (0 = non-admin | 1 = admin)
**Caution: self-building does not have to be used on its own. See the [Alternative Architectures](alternative-architectures.md) page.**
The playbook supports self-building of various components, which don't have a container image for your architecture (see the [container images we use](container-images.md)). For `amd64`, self-building is not required.
For other architectures (e.g. `arm32`, `arm64`), ready-made container images are used when available. If there's no ready-made image for a specific component and said component supports self-building, an image will be built on the host. Building images like this takes more time and resources (some build tools need to get installed by the playbook to assist building).
To make use of self-building, you don't need to do anything besides change your architecture variable (e.g. `matrix_architecture: arm64`). If a component has an image for the specified architecture, the playbook will use it directly. If not, it will build the image on the server itself.
Note that **not all components support self-building yet**.
List of roles where self-building the Docker image is currently possible:
- `matrix-synapse`
- `matrix-synapse-admin`
- `matrix-client-element`
- `matrix-client-hydrogen`
- `matrix-registration`
- `matrix-coturn`
- `matrix-corporal`
- `matrix-ma1sd`
- `matrix-mailer`
- `matrix-bridge-appservice-irc`
- `matrix-bridge-appservice-slack`
- `matrix-bridge-appservice-webhooks`
- `matrix-bridge-mautrix-facebook`
- `matrix-bridge-mautrix-hangouts`
- `matrix-bridge-mautrix-googlechat`
- `matrix-bridge-mautrix-telegram`
- `matrix-bridge-mautrix-signal`
- `matrix-bridge-mautrix-whatsapp`
- `matrix-bridge-mx-puppet-skype`
- `matrix-bot-mjolnir`
- `matrix-bot-matrix-reminder-bot`
- `matrix-email2matrix`
Adding self-building support to other roles is welcome. Feel free to contribute!
If you'd like **to force self-building** even if an image is available for your architecture, look into the `matrix_*_self_build` variables provided by individual roles.
- If your server federates with others, make sure to **leave any federated rooms before nuking your Matrix server's data**. Otherwise, the next time you set up a Matrix server for this domain (regardless of the installation method you use), you'll encounter trouble federating.
- If you have some trouble with your installation, you can just [re-run the playbook](installing.md) and it will try to set things up again. **Uninstalling and then installing anew rarely solves anything**.
-----------------
## Uninstalling using a script
Installing places a `/usr/local/bin/matrix-remove-all` script on the server.
You can run it to to have it uninstall things for you automatically (see below). **Use with caution!**
## Uninstalling manually
If you prefer to uninstall manually, run these commands (most are meant to be executed on the Matrix server itself):
- ensure all Matrix services are stopped: `ansible-playbook -i inventory/hosts setup.yml --tags=stop` (if you can't get Ansible working to run this command, you can run `systemctl stop 'matrix*'` manually on the server)
- delete the Matrix-related systemd `.service` and `.timer` files (`rm -f /etc/systemd/system/matrix*.{service,timer}`) and reload systemd (`systemctl daemon-reload`)
- delete some helper scripts (`rm -f /usr/local/bin/matrix*`)
- delete some cached Docker images (`docker system prune -a`) or just delete them all (`docker rmi $(docker images -aq)`)
- delete the Docker networks: `docker network rm matrix matrix-coturn` (might have been deleted already if you ran the `docker system prune` command)
- uninstall Docker itself, if necessary
- delete the `/matrix` directory (`rm -rf /matrix`)
**Note**: `<your-username>` is just a plain username (like `john`), not your full `@<username>:<your-domain>` identifier.
**You can then log in with that user** via the Element service that this playbook has created for you at a URL like this: `https://element.<domain>/`.
## Option 2 (if you are using an external Postgres server):
You can manually generate the password hash by using the command-line after **SSH**-ing to your server (requires that [all services have been started](installing.md#starting-the-services)):
and then connecting to the postgres server and executing:
```
UPDATE users SET password_hash = '<password-hash>' WHERE name = '@someone:server.com'
```
where `<password-hash>` is the hash returned by the docker command above.
## Option 3:
Use the Synapse User Admin API as described here: https://github.com/matrix-org/synapse/blob/master/docs/admin_api/user_admin_api.rst#reset-password
This requires an access token from a server admin account. *This method will also log the user out of all of their clients while the other options do not.*
If you didn't make your account a server admin when you created it, you can use the `/usr/local/bin/matrix-change-user-admin-status` script as described in [registering-users.md](registering-users.md).
### Example:
To set @user:domain.com's password to `correct_horse_battery_staple` you could use this curl command:
This directory contains sample files that show you how to do reverse-proxying using Apache.
This is for when you wish to have your own Apache webserver sitting in front of Matrix services installed by this playbook.
See the [Using your own webserver, instead of this playbook's nginx proxy](../../docs/configuring-playbook-own-webserver.md) documentation page.
To use your own Apache reverse-proxy, you first need to disable the integrated nginx server.
You do that with the following custom configuration (`inventory/host_vars/matrix.<your-domain>/vars.yml`):
```yaml
matrix_nginx_proxy_enabled: false
```
You can then use the configuration files from this directory as an example for how to configure your Apache server.
**NOTE**: this is just an example and may not be entirely accurate. It may also not cover other use cases (enabling various services or bridges requires additional reverse-proxying configuration).
This directory contains sample files that show you how to do reverse-proxying using Caddy2.
## Config
| Variable | Function |
| ------------------ | -------- |
| tls your@email.com | Specify an email address for your [ACME account](https://caddyserver.com/docs/caddyfile/directives/tls) (but if only one email is used for all sites, we recommend the email [global option](https://caddyserver.com/docs/caddyfile/options) instead) |
| tls | To enable [tls](https://caddyserver.com/docs/caddyfile/directives/tls) support uncomment the lines for tls |
| Dimension | To enable Dimension support uncomment the lines for Dimension and set your data |
| Jitsi | To enable Jitsi support uncomment the lines for Jitsi and set your data |
This directory contains sample files that show you how to do reverse-proxying using HAproxy.
This is for when you wish to have your own HAproxy instance sitting in front of Matrix services installed by this playbook.
See the [Using your own webserver, instead of this playbook's nginx proxy](../../docs/configuring-playbook-own-webserver.md) documentation page.
To use your own HAproxy reverse-proxy, you first need to disable the integrated Nginx server.
You do that with the following custom configuration (`inventory/host_vars/matrix.<your-domain>/vars.yml`):
```yaml
matrix_nginx_proxy_enabled: false
```
You can then use the configuration files from this directory as an example for how to configure your HAproxy reverse proxy.
**NOTE**: this is just an example and may not be entirely accurate. It may also not cover other use cases or performance needs.
### Configuration
HAproxy, unlike Apache, Nginx and others, does not provide you with a webserver to serve static files (i.e., `/.well-known/` directory). For this reason, in this folder you can find an example on how to use HAproxy together with a simple Nginx container whose only task is to serve those files.
* Build the Docker image. `docker build -t local/nginx .`
* Start the container. `docker-compose up -d`. Note that if you want to run Nginx on a different port, you will have to change the port both in the `docker-compose.yml` and in `haproxy.cfg`.
* If you don't want to use a wildcard certificate, you will need to modify the corresponding line in the HTTPS frontent and add the paths of all the specific certificates (as for the commented example in `haproxy.cfg`).
* Start HAproxy with the proposed configuration.
Some files were not shown because too many files have changed in this diff
Show More
loooph
Slavi Pantaleev
Luke
Michael Collins
Aaron R
Samonitari
Kim Brose
Toni Spets
Alexandar Mechev
Oli
Jan
Colin Shea
WobbelTheBear
sakkiii
Krisztian Szegi
Catalan Lover
Hagen
sakkiii
Wolfgang Winter
Joseph Walton-Rivers
Thom Wiggers
nono
AtomHare
pushytoxin
Wolfgang Winter
Dan Arnfield
Jaffex
Sebastian Gumprich
Sebastian Gumprich
0xLAITH
Nate
Hardy Erlinger
maximilianschmelzer
Toorero
JokerGermany
Richard Meyer
Janar Juusu
Markus
Sergei Shikalov
Neutron
Stuart Mumford
sak
oxmie
Davy Landman
Michael Sasser
hanthor
Marcel Ackermann
N07070
Blaž Tomažič
SkepticalWaves
rakshazi
Thomas Kühne
Raymond Coetzee
Jez Cope
BG
Ahmad Haghighi
ewang
Eagle-251
throwawayay
Rafael Girão
Tobias K
Béla Becker
benkuly
teutat3s
Marcus
foxcris
Michael
Christoph Johannes Kleine
transcaffeine
Thom Wiggers
chagai95
Yannick Goossens
Alexandros Afentoulis
Pablo Montepagano
Markus Ullmann
SierraKiloBravo
tctovsli
Marc Leuser
Germain
Martha Sokolska
Cody Neiman
Joel Bennett
Peetz0r
efraimbart
Yan
buxel
Stuart Thomson
Paul Tötterman
Julian Foad
o8F0LY
phirz
T. Küchel
Marcel Partap
Prasiddh Pooskur
Panagiotis Georgiadis
Will
Agustin Ferrario
Sabine Laszakovits
0hlov3
laszabine
David Gnedt
louis
Tomas Strand
John Goerzen
Mike
anonym
João Marques
Scott Crossen
Eneko Nieto
Ivar Troost
Tobias Küchel
jgbresson
Fanch
dhose
Panagiotis Vasilopoulos
Clement Renaud
Chris van Dijk
Daniel Wiegreffe
Max Klenk
jens quade
0hlov3
Daniel Wiegreffe
0hlov3
Dominik Zajac
Aki Salminen
Justin Croonenberghs
MatthewCroughan
merklaw
Dennis Ciba
vaivars
vractal
Benjamin Fichtner
Matthias
rebru
Tommy Kelly
TwoTwenty
Rónán Duddy
hungrymonkey
Matt Cengia
aa1874
bertiebaggio
Julian Strobl
shadow
dasTholo
Hugues Morisset
Tulir Asokan
jdreichmann
Pablo
kristbaum
Ugurtan
joao-p-marques
Kevin Lanni
Christian Wolf
Fanch
Horvath Gergely
André Sterba
GuillauG
Mickaël Cornière
Rodrigo Belem
Tom
Udo Rader
skoal
Sean O'Neil
Alin Trăistaru
PlanetSmasher
mooomooo
ayounggun
Stefan Warnat
iLyas Bakouch
Sathis
orange
Daniel Hoffend
Lee Verberne
Zach Mertes
prasket
prasket
Björn Marten
Lyubomir Popov
Gergely Horváth
Hanno J. Gödecke
DreamsVoid
Daniel Løvbrøtte Olsen
Logan Pulley
Yuri Moens
benkuly
Michael Haak
recklesscoder
fnoah
Noah Fleischmann
Paul Tötterman
Ciarán Ainsworth
Arehandoro
gusttt
liquidat
muccid
sudneo
Eduardo Beltrame
kingoftheconnors
Oleg Fiksel
p5t2vspoqqw
microchipster
Esko
Dan Arnfield
Thomas Kuehne
Michael Haak
Jason Locklin
inthewaves
Brendan Abolivier
Hugues De Keyzer
Sylvia van Os
Lyubomir Popov
Ciaran Ainsworth
NullIsNot0
Alexander Acevedo
Borjan Tchakaloff
Lorrin Nelson
Plailect
Edgars Voroboks
paulbdavis
Lionstiger
frlae
Jan Christian Grünhage
anadahz
haslersn
Thomas vO
TheForcer
Devr
Slavi Pantaleev