coop-cloud/mailman3
1
0
Fork 0
Code Issues 9 Pull Requests Releases Wiki Activity

Mailu/Mailman3 integration terror thread #2

New Issue
Closed
opened 2021-05-13 08:50:43 +00:00 by decentral1se · 16 comments
decentral1se commented 2021-05-13 08:50:43 +00:00
Owner
Copy Link

A home for all the mess. Best keep it under this repo.

Also see https://git.autonomic.zone/coop-cloud/mailu/issues/1.

A home for all the mess. Best keep it under this repo. Also see https://git.autonomic.zone/coop-cloud/mailu/issues/1.
decentral1se commented 2021-05-13 09:15:09 +00:00
Author
Owner
Copy Link

Current status:

image

(This helped!)

Current status: ![image](/attachments/29a6c1d6-f903-4799-9ea5-b5fda492dd7c) ([This](https://anarc.at/services/mail/#dns) helped!)
image.png
65 KiB
decentral1se commented 2021-05-14 08:04:21 +00:00
Author
Owner
Copy Link

OK, my current idea is then:

  • Stand the mailman3 instance up

  • Add DNS entries for the incoming mail config

image

  • Config the shit out of the mailman3 instance to wire up outgoing mail

  • Test, test, test

OK, my current idea is then: - [ ] Stand the mailman3 instance up - [ ] Add DNS entries for the incoming mail config ![image](/attachments/b3912452-bad0-422e-8137-b4aa5681aa3e) - [ ] Config the shit out of the mailman3 instance to wire up outgoing mail - [ ] Test, test, test
image.png
66 KiB
decentral1se commented 2021-05-14 08:53:08 +00:00
Author
Owner
Copy Link

RE: "Add DNS entries for the incoming mail config" in https://git.autonomic.zone/coop-cloud/mailman3/issues/2#issuecomment-5712 above:

image

RE: "Add DNS entries for the incoming mail config" in https://git.autonomic.zone/coop-cloud/mailman3/issues/2#issuecomment-5712 above: ![image](/attachments/14372f50-f71f-45fe-8e5f-39f597b003ec)
image.png
33 KiB
decentral1se commented 2021-05-14 09:25:59 +00:00
Author
Owner
Copy Link

Was looking at https://github.com/bokysan/docker-postfix for d0ae866623/compose.yml (L37-L40).

Was looking at https://github.com/bokysan/docker-postfix for https://git.autonomic.zone/coop-cloud/mailman3/src/commit/d0ae86662345ff0cc352c18bd4a9eee38cef54b0/compose.yml#L37-L40.
decentral1se commented 2021-05-14 09:31:40 +00:00
Author
Owner
Copy Link

Potential plan to avoid the additional postfix configuration:

image

May need to re-work the mailu postfix configuration but maybe that is good too!

Potential plan to avoid the additional postfix configuration: ![image](/attachments/e22a6bcd-ae3f-48cf-b677-cf0fe48a54da) May need to re-work the mailu postfix configuration but maybe that is good too!
image.png
22 KiB
decentral1se commented 2021-05-14 15:53:07 +00:00
Author
Owner
Copy Link

OK, added the DNS entries (MX, SPF). Managed to stand-up the mailman3 install also (taken down for now)! However, the REST API connection is not working. That is the next thing to fix. Then I can finally start debugging incoming/outgoing mail.

OK, added the DNS entries (MX, SPF). Managed to stand-up the mailman3 install also (taken down for now)! However, the REST API connection is not working. That is the next thing to fix. Then I can finally start debugging incoming/outgoing mail.
decentral1se commented 2021-05-16 16:03:54 +00:00
Author
Owner
Copy Link

abra app lists.autonomic.zone deploy now works!

The next step is to wire up the shared network and volumes for mailman-core to be able to speak to the our mailu postfix container (from 02e4559768/mailman-extra.cfg.tmpl (L10) to a23e9fc40b/compose.yml (L113-L119) afaiu).

For the shared network, I am hoping I can put the core service in the mailman config on the mailu default overlay network and declare it as external from the mailman side.

➜  mailu (main) ✔ docker network ls -f name=mail
NETWORK ID     NAME                          DRIVER    SCOPE
vjyo6enrhezh   mail_autonomic_zone_default   overlay   swarm

Annnd for the shared volume, I am not sure, but using this external trick to load the mailman-core postfix mapping generated files into the mailu postfix container will hopefully work.

The final piece of the puzzle is transport_maps/local_recipient_maps/relay_domains overridden in the mailu configured postfix to push stuff over to the mailman LMTP endpoint. I'm looking for ways to hook in via https://github.com/Mailu/Mailu/tree/master/core/postfix/conf.

`abra app lists.autonomic.zone deploy` now works! The next step is to wire up the shared network and volumes for mailman-core to be able to speak to the our mailu postfix container (from https://git.autonomic.zone/coop-cloud/mailman3/src/commit/02e4559768dbdcbdd1877bf19d60d3219c780c5a/mailman-extra.cfg.tmpl#L10 to https://git.autonomic.zone/coop-cloud/mailu/src/commit/a23e9fc40bd914ba431b7cbe41adba1aefd1310e/compose.yml#L113-L119 afaiu). For the shared network, I am hoping I can put the core service in the mailman config on the mailu default overlay network and declare it as external from the mailman side. ``` ➜ mailu (main) ✔ docker network ls -f name=mail NETWORK ID NAME DRIVER SCOPE vjyo6enrhezh mail_autonomic_zone_default overlay swarm ``` Annnd for the shared volume, I am not sure, but using this external trick to load the mailman-core postfix mapping generated files into the mailu postfix container will hopefully work. The final piece of the puzzle is transport_maps/local_recipient_maps/relay_domains overridden in the mailu configured postfix to push stuff over to the mailman LMTP endpoint. I'm looking for ways to hook in via https://github.com/Mailu/Mailu/tree/master/core/postfix/conf.
decentral1se commented 2021-05-16 20:48:13 +00:00
Author
Owner
Copy Link

The final piece of the puzzle is transport_maps/local_recipient_maps/relay_domains overridden in the mailu configured postfix to push stuff over to the mailman LMTP endpoint. I'm looking for ways to hook in via https://github.com/Mailu/Mailu/tree/master/core/postfix/conf.

See https://mailu.io/1.7/faq.html?highlight=override#how-can-i-override-settings.

> The final piece of the puzzle is transport_maps/local_recipient_maps/relay_domains overridden in the mailu configured postfix to push stuff over to the mailman LMTP endpoint. I'm looking for ways to hook in via https://github.com/Mailu/Mailu/tree/master/core/postfix/conf. See https://mailu.io/1.7/faq.html?highlight=override#how-can-i-override-settings.
decentral1se commented 2021-05-18 19:57:38 +00:00
Author
Owner
Copy Link
- the mailu side postfix overrides and shared volume (haven't tested this *at all*): https://git.autonomic.zone/coop-cloud/mailu/src/commit/98c5bbc95f3aefe1f6535c1b763d103bdc780947/compose.mailman.yml - the mailman side shared overlay network to get smtp connections from the mailu postfix configuration (also haven't tested this *at all*): https://git.autonomic.zone/coop-cloud/mailman3/src/commit/3266e9ffbcd51fb8adff6953da63680422373b24/compose.yml#L12
decentral1se commented 2021-05-18 20:04:26 +00:00
Author
Owner
Copy Link

To start testing this, we need to (I think!):

  1. abra app lists.autonomic.zone deploy

  2. check what is generated in /opt/mailman/core/var/data/postfix_lmtp and take note of the hostname on the mailman stack

  3. re-deploy mail.autonomic.zone with COMPOSE_FILE="compose.yml:compose.mailman.yml" and make sure it comes up

  4. Drop into a shell on the mailu side and make the external volume is available in /opt/mailman/ and the hostname mailman-core in pingable. postfix should also have loaded the /overrides/... configuration. Ensure the hostname generated in the /opt/mailman/core/var/data/ files (see step 2) is mailman-core or some other pingable address (ensures mailu can pass mails to mailman LTMP)

  5. Drop into a shell on the mailman side and make sure $SMTP_NETWORK in pingable (ensures we can send outgoing mail)

To start testing this, we need to (I think!): 1. `abra app lists.autonomic.zone deploy` 1. check what is generated in `/opt/mailman/core/var/data/postfix_lmtp` and take note of the hostname on the mailman stack 1. re-deploy `mail.autonomic.zone` with `COMPOSE_FILE="compose.yml:compose.mailman.yml"` and make sure it comes up 1. Drop into a shell on the mailu side and make the external volume is available in `/opt/mailman/` and the hostname `mailman-core` in pingable. postfix should also have loaded the /overrides/... configuration. Ensure the hostname generated in the `/opt/mailman/core/var/data/` files (see step 2) is `mailman-core` or some other pingable address (ensures mailu can pass mails to mailman LTMP) 1. Drop into a shell on the mailman side and make sure `$SMTP_NETWORK` in pingable (ensures we can send outgoing mail)
decentral1se commented 2021-05-21 13:00:56 +00:00
Author
Owner
Copy Link
  • Fix this archiver issue:
May 21 12:24:11 2021 (190) HyperKitty failure on http://mailman-web:8000/hyperkitty/api/mailman/urls: <html><title>Forbidden</title><body>
                <h1>Access is forbidden</h1><p>Please check the IP addresses
                 assigned to MAILMAN_ARCHIVER_FROM in the settings file.
                </p></body></html> (403)

  • Fix the double /etc/hosts entry

  • Document the madness

  • Revisit if any of this is really bad

- Fix this archiver issue: ``` May 21 12:24:11 2021 (190) HyperKitty failure on http://mailman-web:8000/hyperkitty/api/mailman/urls: <html><title>Forbidden</title><body> <h1>Access is forbidden</h1><p>Please check the IP addresses assigned to MAILMAN_ARCHIVER_FROM in the settings file. </p></body></html> (403) ``` - Fix the double `/etc/hosts` entry - Document the madness - Revisit if any of this is really bad
decentral1se commented 2021-05-21 15:47:39 +00:00
Author
Owner
Copy Link

Remaining issues:

  • Fix the double /etc/hosts entry problem
    • [2021-05-21 15:45:44 +0000] [48] [ERROR] Can't connect to ('10.0.52.25 192.168.203.47', 8001)
  • Revisit if any of this is really bad
  • Check the workflow of stack rm'ing and how that works with shared network/volumes
Remaining issues: - Fix the double /etc/hosts entry problem - `[2021-05-21 15:45:44 +0000] [48] [ERROR] Can't connect to ('10.0.52.25 192.168.203.47', 8001)` - Revisit if any of this is really bad - Check the workflow of `stack rm`'ing and how that works with shared network/volumes
decentral1se commented 2021-05-23 08:58:30 +00:00
Author
Owner
Copy Link

I realised that the relay domain is not even needed for our current frankenstein setup. Since we load in the maps, the mailu postfix knows exactly where to pass things (and thats why it was trying port 25 on the incoming relaying at first!). I was reading https://mailu.io/master/webadministration.html#relayed-domains.

I realised that the relay domain is not even needed for our current frankenstein setup. Since we load in the maps, the mailu postfix knows exactly where to pass things (and thats why it was trying port 25 on the incoming relaying at first!). I was reading https://mailu.io/master/webadministration.html#relayed-domains.
decentral1se commented 2021-05-23 09:46:27 +00:00
Author
Owner
Copy Link

I attempted to drop the shared volume and use the relay domain directly but ran into https://github.com/Mailu/Mailu/issues/1815. So, we need the shared volume to tell the mailu postfix that it should use LMTP to do the communication, not SMTP.

I attempted to drop the shared volume and use the relay domain directly but ran into https://github.com/Mailu/Mailu/issues/1815. So, we need the shared volume to tell the mailu postfix that it should use LMTP to do the communication, not SMTP.
decentral1se commented 2021-05-23 10:03:45 +00:00
Author
Owner
Copy Link

2f27f298f9 "solves" the double entry issue on the core /etc/hosts/ side.

https://git.autonomic.zone/coop-cloud/mailman3/commit/2f27f298f95a48df284b1a4f4bcd2b6240f433cd "solves" the double entry issue on the core `/etc/hosts/` side.
decentral1se commented 2021-05-23 10:24:04 +00:00
Author
Owner
Copy Link

🙈

🙈
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No Label
Milestone
No items
No Milestone
Assignees
Clear assignees
3wordchant aadil abra-bot ammaratef45 Apfelwurm appletalk arjan basebuilder Brooke carla cas codegod100 coopcloud decentral1se fauno flancian Frando iexos javielico jmakdah2 kawaiipunk knoflook lambdabundesverband mac-chaffee marlon mayel mirsal moritz nicksellen notplants p4u1 pau renovate-bot ripclap rix sef simon sixsmith stevensting tobias trav val virtualboys wolcen wykwit xynosis yksflip
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: coop-cloud/mailman3#2
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?