| Time |
Nick |
Message |
| 00:37 |
|
sandbergja joined #evergreen |
| 06:01 |
pinesol |
News from qatests: Testing Success <http://testing.evergreen-ils.org/~live> |
| 07:07 |
|
rfrasur joined #evergreen |
| 07:08 |
|
rjackson_isl joined #evergreen |
| 07:49 |
|
collum joined #evergreen |
| 08:00 |
|
tlittle joined #evergreen |
| 08:19 |
|
collum joined #evergreen |
| 08:22 |
|
collum_ joined #evergreen |
| 08:38 |
|
mantis1 joined #evergreen |
| 08:40 |
|
_bott_ joined #evergreen |
| 08:46 |
|
mmorgan joined #evergreen |
| 09:02 |
|
Dyrcona joined #evergreen |
| 09:15 |
dbs |
Hmm. Any idea why we're starting to see multiple copies of the same report with identical run_time and scheduled_wait_time values in reporter.pending_reports? |
| 09:16 |
dbs |
(with the somewhat comical outcome of getting dozens of copies of notification for the same report when it does run) |
| 09:22 |
mmorgan |
dbs: Do the duplicates also show up in reporter.schedule? |
| 09:25 |
|
dbwells joined #evergreen |
| 09:25 |
|
dbwells joined #evergreen |
| 09:30 |
|
_bott_ joined #evergreen |
| 09:32 |
dbs |
mmorgan: yes they do - according to SELECT report, email, runner, run_time, start_time FROM reporter.schedule WHERE complete_time IS NULL ORDER BY 1, 4; |
| 09:35 |
dbs |
here's the start of the output, minus the email column for privacy: https://pastebin.com/L2E26pHb |
| 09:37 |
dbs |
we run clark with a concurrency of 2, IIRC |
| 09:45 |
mmorgan |
dbs: Can you run your query above with the addition of id and name? |
| 09:47 |
|
yboston joined #evergreen |
| 09:49 |
dbs |
mmorgan: sure: https://pastebin.com/wHt8CthC |
| 09:50 |
dbs |
In the interim, I cloned our reporter.schedule table and then deleted all of the duplicates from the real reporter.schedule table. So that report is coming from the cloned table (joined to reporter.report for the name column) as it was before my dupe deletion. |
| 09:53 |
mmorgan |
dbs: is reporter.schedule.folder the same in the duplicates, too? |
| 09:54 |
dbs |
mmorgan: yes it is. |
| 09:55 |
dbs |
everything is identical except for the id |
| 09:55 |
mmorgan |
The client won't let you do that, so not a human causing the duplicates. But maybe you knew that already :) |
| 09:57 |
dbs |
Heh, yes, I was pretty sure it was not a human :) |
| 09:58 |
dbs |
For the logs, my delete duplicate reports statement was: |
| 09:58 |
dbs |
BEGIN; DELETE FROM reporter.schedule WHERE complete_time IS NULL AND report IN (SELECT report FROM ( SELECT report, ROW_NUMBER() OVER w AS rnum FROM reporter.schedule WINDOW w AS (PARTITION BY email, runner, run_time, start_time ORDER BY report)) t WHERE t.rnum > 1); |
| 10:02 |
|
mmorgan1 joined #evergreen |
| 10:03 |
dbs |
Bmagic++ # used your 3.4.0 Docker image over the weekend to do some debugging - very handy - also works in podman under Fedora |
| 10:04 |
|
sandbergja joined #evergreen |
| 10:25 |
|
mmorgan joined #evergreen |
| 10:38 |
|
_bott_ joined #evergreen |
| 10:45 |
|
_bott_ joined #evergreen |
| 10:58 |
* dbs |
revisits that DELETE statement - no, no, that wasn't quite right |
| 11:02 |
|
mantis1 joined #evergreen |
| 11:12 |
|
sandbergja joined #evergreen |
| 11:18 |
csharp |
dbs: I don't have any insight, but I have seen that happen several times in PINES |
| 11:22 |
Bmagic_ |
dbs: sweet! |
| 11:28 |
dbs |
My hypothesis is that when our Perl OpenSRF services crash due to the PG postmaster terminating all connections, Clark happily connects to the database and tries to do its work, fails at some point, but not before scheduling the next report |
| 11:29 |
dbs |
And that happens each time Clark connects and fails to run a given report while the OpenSRF services are in that state, doubling and redoubling the scheduled reports |
| 11:47 |
dbs |
Looking around line 300 of clark-kent.pl, though, I'm not sure how that happens, as the INSERT happens in a transaction that should also set the complete_time. Hrm. |
| 11:49 |
|
mantis1 joined #evergreen |
| 12:08 |
|
jihpringle joined #evergreen |
| 12:39 |
|
khuckins joined #evergreen |
| 12:44 |
|
_bott_ joined #evergreen |
| 12:53 |
|
cmalm joined #evergreen |
| 12:57 |
|
mantis1 joined #evergreen |
| 12:58 |
|
mantis2 joined #evergreen |
| 14:21 |
|
_bott_ joined #evergreen |
| 14:35 |
|
_bott_ joined #evergreen |
| 14:40 |
|
_bott_ joined #evergreen |
| 14:48 |
csharp |
seeing some log errors on our test server that don't make sense to me: 2020-01-13 14:38:44 next-brick01-head open-ils.cstore: [ERR :20970:oils_sql.c:6993:1578914702188581199] open-ils.cstore ERROR No "datatype" attribute for field "hopeless_prone" |
| 14:48 |
csharp |
we've applied the hopeless holds branch and are planning to go live with it |
| 14:49 |
csharp |
fm_IDL.xml hwas this: <field name="hopeless_prone" reporter:datatype="bool"/> |
| 14:49 |
csharp |
so I don't know what the code is looking for if not that |
| 14:50 |
mmorgan |
csharp: I believe hopeless_prone would be an attribute of a copy status |
| 14:50 |
csharp |
right |
| 14:50 |
* Dyrcona |
sighs. |
| 14:50 |
csharp |
https://git.evergreen-ils.org/?p=Evergreen.git;a=blob;f=Open-ILS/src/c-apps/oils_sql.c;h=ac2f054cb6594068d389580134d4f4d2962c3ecb;hb=HEAD#l6985 is the code that generates that error |
| 14:51 |
csharp |
(looks like the first function should say "ERROR No "primitive" attribute..." - probably worth a bitesize bug |
| 14:53 |
csharp |
also seeing these: |
| 14:53 |
csharp |
2020-01-13 14:19:35 next-brick01-head open-ils.cstore: [ERR :18861:oils_sql.c:6993:157891470220628169] open-ils.cstore ERROR No "datatype" attribute for field "org_unit" |
| 14:53 |
csharp |
2020-01-13 14:20:25 next-brick01-head open-ils.cstore: [ERR :18861:oils_sql.c:6993:157891470220626298] open-ils.cstore ERROR No "datatype" attribute for field "hopeless_date" |
| 14:54 |
csharp |
not sure which "org_unit" is referred to |
| 14:54 |
csharp |
<field reporter:label="Hopeless Date" name="hopeless_date" reporter:datatype="timestamp"/> - so that has a datatype too |
| 14:57 |
phasefx_ |
csharp: the two fm_IDL.xml files are in sync? |
| 15:04 |
csharp |
phasefx: yes - I've symlinked the reports one to the conf one |
| 15:05 |
phasefx |
csharp: roger roger; not familiar with that error |
| 15:05 |
mmorgan |
autogen? |
| 15:05 |
csharp |
I've run autogen, but can again :-/ |
| 15:05 |
phasefx |
with cstore catching it, I'd expect something deeper |
| 15:30 |
|
mantis2 left #evergreen |
| 16:05 |
Bmagic |
dbs: you prompted me to build a 3.4.1 container. Posted it |
| 16:15 |
|
jihpringle joined #evergreen |
| 16:33 |
|
jihpringle joined #evergreen |
| 16:39 |
csharp |
Bmagic++ |
| 16:39 |
Bmagic |
:) |
| 17:04 |
|
sandbergja_ joined #evergreen |
| 17:07 |
|
mmorgan left #evergreen |
| 17:10 |
|
yboston_ joined #evergreen |
| 17:23 |
|
_bott_ joined #evergreen |
| 17:38 |
|
khuckins joined #evergreen |
| 17:49 |
|
sandbergja_ joined #evergreen |
| 18:01 |
pinesol |
News from qatests: Testing Success <http://testing.evergreen-ils.org/~live> |
| 18:06 |
|
stephengwills left #evergreen |
| 18:15 |
|
sandbergja_ joined #evergreen |
| 19:25 |
|
sandbergja_ joined #evergreen |
| 19:29 |
|
genpaku joined #evergreen |
| 19:33 |
|
jvwoolf1 joined #evergreen |
| 21:21 |
|
_bott_ joined #evergreen |
| 21:58 |
|
sandbergja joined #evergreen |
