| Time |
Nick |
Message |
| 07:17 |
|
rjackson_isl_hom joined #evergreen |
| 07:53 |
|
rfrasur joined #evergreen |
| 08:24 |
|
collum joined #evergreen |
| 08:28 |
|
jvwoolf joined #evergreen |
| 08:29 |
|
Dyrcona joined #evergreen |
| 08:34 |
|
mantis joined #evergreen |
| 08:39 |
|
Dyrcona joined #evergreen |
| 08:39 |
|
mmorgan joined #evergreen |
| 08:47 |
Dyrcona |
Hm. It looks like one has to reboot after updating the ejabberd config on Ubuntu 22.04. |
| 08:49 |
Dyrcona |
I should check if there's another service that needs a kick, too. |
| 09:04 |
Dyrcona |
Nothing in the release notes and nothing on Launchpad that looks relevant. When I setup ejabberd according to the README instructions, systemctl hangs on the start of ejabberd. This was what I meant when I said I hadn't gotten ejabberd to work the other day. |
| 09:04 |
Dyrcona |
The ejabberd logs are full of messages. |
| 09:05 |
Dyrcona |
If I reboot the virtual machine, ejabberd comes right up with the new configuration. |
| 09:05 |
Dyrcona |
I tried search the web for some of the log messages, but nothing relevant came up there either. |
| 09:09 |
Dyrcona |
Ugh. Hit a key in this window that I meant for the browser, and it cleared the chat frame. |
| 09:11 |
Dyrcona |
I suppose that I should try and resolve the autorecconf messages as well.... |
| 09:57 |
jeff |
if this a scenario where you have renamed the host (intentionally or as a side effect of getting a new hostname via dhclient/etc) and your erlang node name has changed? common on virtual machines, but you probably already know that. |
| 09:58 |
jeff |
usually our method is to set ERLANG_NODE in /etc/default/ejabberd, but I think recent packaged installs use a reasonable hostname-independent node name. |
| 09:58 |
Dyrcona |
jeff: No. It isn't that scenario. I made the VM, installed the prerequisites, reconfigured ejabberd, i.e. the typical steps that don't require a restart. |
| 09:59 |
jeff |
of course, THAT could be a change that's causing you issues, I don't remember how ERLANG_NODE interacts with s2s communication, which you may rely on. |
| 09:59 |
Dyrcona |
I mean reboot. |
| 09:59 |
Dyrcona |
After I get some of the other things worked out, I'm build a fresh VM and look into the ejabberd stuff some more. |
| 10:00 |
jeff |
I may have run into another scenario where ejabberdctl hangs. It may have been an apparmor interaction. |
| 10:00 |
Dyrcona |
I can say the autoconf changes work, and they work on Debian 9, Stretch, as well as Ubuntu 18.04, so I think that's done. |
| 10:01 |
Dyrcona |
jeff: Yes, I've seen a bug from last year on Launchpad. |
| 10:01 |
jeff |
Oh, I was thinking of an Ansible issue I encountered, where I also scribbled a note "old bug, worth revisiting?" for bug 1832408 |
| 10:01 |
pinesol |
Launchpad bug 1832408 in OpenSRF "Cannot (or should not) use root to register ejabberd users" [Undecided,New] https://launchpad.net/bugs/1832408 |
| 10:02 |
Dyrcona |
jeff: bug 1934508 |
| 10:02 |
pinesol |
Launchpad bug 1934508 in ejabberd (Ubuntu) "Ejabberd fails with latest apparmor profile on Ubuntu 21.04" [Undecided,New] https://launchpad.net/bugs/1934508 |
| 10:02 |
jeff |
The Ansible issue with ejabberd was a bug in the community.general, ejabberd_user collection. My notes on that say: |
| 10:02 |
jeff |
> recent changes broke "exists" and "changed" detection, because it's no longer testing the return value of the ejabberdctl command, so it always thinks that things exist and always thinks that user passwords need to be set, even if they are already set to the chosen value. |
| 10:02 |
jeff |
(so, probably unrelated to your issue) |
| 10:03 |
Dyrcona |
I want to investigate a segfault from testing the other day. I made a typo when doing opensrf.math add and srfsh segfaulted. |
| 10:04 |
Dyrcona |
I'd rather not say, "Reboot the server." in the installation instructions, but if that is what it comes down to, then OK. :) |
| 10:08 |
Dyrcona |
`make check` still works on Debian 9, so I assume it will work everywhere else since I've tested it on all releases of Ubuntu and the other Debian releases are closely related. |
| 10:12 |
Dyrcona |
So, it looks like if you make a request to a non-existent service from srfsh on Ubuntu 22.04, srfsh segfaults. A bogus method called on a real service returns a 404. On Debian 9, the former doesn't crash and says 'No data received from server.' |
| 10:18 |
Dyrcona |
Ok. It looks like the router and some other services also crash. |
| 10:31 |
Dyrcona |
OK. The router, etc., are not crashing now that I'm trying to make them crash. |
| 10:33 |
Dyrcona |
srfsh still crashes |
| 11:11 |
|
jvwoolf joined #evergreen |
| 11:32 |
|
rjackson_isl_hom joined #evergreen |
| 12:16 |
|
jihpringle joined #evergreen |
| 12:18 |
Dyrcona |
It's always bothersome when someone opens a ticket regarding something that was fixed months ago, particularly when the ticket comes on the heels of other changes. It makes me stop and double check that the change is still in place. |
| 12:34 |
|
collum joined #evergreen |
| 12:36 |
|
collum joined #evergreen |
| 13:08 |
|
jvwoolf1 joined #evergreen |
| 13:31 |
csharp_ |
Dyrcona: sometimes the /run vs. /var/run location of the pid is an issue when systemctl hangs - maybe that happens at reboot differently? |
| 13:36 |
Dyrcona |
csharp_: It could be, but I'm pretty sure that I stopped ejabberd before making configuration changes. I have restarted ejabberd with systemctl after rebooting and that worked OK. I haven't tried a stop followed by a start. |
| 13:37 |
Dyrcona |
Hmmm. Stop followed by start seems to do it. |
| 13:40 |
Dyrcona |
Apr 28 13:39:56 devjam kernel: [17602.611246] audit: type=1400 audit(1651167596.601:1574): apparmor="ALLOWED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/ejabberdctl" name="run/systemd/notify" pid=15325 comm="2_scheduler" requested_mask="w" denied_mask="w" fsuid=115 ouid=0 |
| 13:45 |
Dyrcona |
So, further poking suggests that ejabberd is starting but not properly notifying ejabberd that it has started. |
| 13:45 |
Dyrcona |
Sorry, that second ejabberd should be systemd. |
| 13:52 |
Dyrcona |
Yeah, that seems to be what is happening. systemctl complains of a tiemout, but ejabberd is running, and osrf services start just fine. |
| 13:52 |
Dyrcona |
I've verified that it stops OK, i.e. that this is a new ejabberd instance. |
| 13:53 |
csharp_ |
maybe the ejabberd config file and systemd service files disagree about something? |
| 13:53 |
Dyrcona |
could be. |
| 13:53 |
Dyrcona |
I'm trying a restart, now, and it seems to be hanging this time. |
| 13:59 |
Dyrcona |
Interesting. If I stop ejabberd with ejabberdctl. It restarts automatically in a few seconds. |
| 14:00 |
Dyrcona |
The same does not happen if I stop it with systemctl. |
| 14:18 |
Dyrcona |
I am not looking forward to upgrading my laptop... |
| 14:27 |
jeffdavis |
I've always had trouble with ejabberd restart, I just do stop-and-start instead now. |
| 14:29 |
Dyrcona |
jeffdavis: This is the first time I've had problems this bad. It looks like a failure to communicate between ejabberd and systemd, but I haven't figured it out, yet. I've asked over in #ubuntu. |
| 14:46 |
|
jihpringle joined #evergreen |
| 14:56 |
|
rfrasur joined #evergreen |
| 15:10 |
|
collum joined #evergreen |
| 15:14 |
Dyrcona |
Disabling the apparmor profile for usr.sbin.ejabberdctl resolves my ejabberd issue. |
| 15:18 |
Dyrcona |
Should i add the steps to disable the apparmor profile to the README? Should I note that there's a bug on Ubuntu 22.04? |
| 15:23 |
Dyrcona |
bug 1970791 |
| 15:23 |
pinesol |
Launchpad bug 1970791 in ejabberd (Ubuntu) "Starting ejabberd with systemctl "hangs" on Ubuntu 22.04 with apparmor profile" [Undecided,New] https://launchpad.net/bugs/1970791 |
| 15:24 |
JBoyer |
It may be worth saying "you need to disable apparmor for ejabberd until this bug is fixed" |
| 15:25 |
Dyrcona |
JBoyer: I'm considering just putting the steps in the README. |
| 15:25 |
JBoyer |
+1 to including how to disable it, yeah. |
| 15:25 |
Dyrcona |
That's probably an apparmor bug and not really an ejabberd bug. I just put it there because that's where the other, similar bug was. |
| 15:26 |
Dyrcona |
I think I'll recommend this for Ubuntu in general. |
| 15:27 |
Dyrcona |
I still have to figure out the segmentation fault. |
| 15:35 |
Dyrcona |
I wonder how I should add this to the README. Should I make it some kind of note/sidebar type thing? I'm leaning towards doing that right before the step by step instructions begin. |
| 15:41 |
* Dyrcona |
wonders if a code block works inside of and admonition. Guess I can run it through asciidoc and find out. |
| 15:46 |
Dyrcona |
I like it. Looks good when rendered with asciidoc. |
| 15:53 |
Dyrcona |
Disabling the apparmor profile also got rid of some other errors/warnings. |
| 15:57 |
Dyrcona |
Has anyone ever built srfsh for debugging before? I'm not confident that I can find the cause of the segfault without gdb. |
| 16:02 |
Dyrcona |
Hmm... just compiling srfsh is a bit trickier than I thought it would be. |
| 16:06 |
Dyrcona |
libtool..... :/ |
| 16:23 |
|
collum joined #evergreen |
| 17:08 |
|
mmorgan left #evergreen |
| 19:06 |
|
jvwoolf1 left #evergreen |
| 19:28 |
miker |
ORC: looks like commits are back on the menu, boys! |
| 19:38 |
pinesol |
News from commits: Forward port 3.9 upgrade script <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=4d25e4865a35b30db3de06e748c3f11334bc6b14> |
| 19:38 |
pinesol |
News from commits: Base 3.9 release notes <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=824a83aed33c1e7b7e3e354f72f30411c1d6c1e0> |
| 19:38 |
pinesol |
News from commits: Translation updates - po files <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=70f5308b4c961d08e02d93fbdbdaecc7120a6394> |
| 19:38 |
pinesol |
News from commits: Translation updates - newpot <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=77c87edab18050b4736ed72b85bca01c4aa4af47> |
| 19:38 |
pinesol |
News from commits: Correct jacket uploader upgrade script <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=42a6947407a898e8a02fca1f3bce3a65fd917c36> |
