| 12:00 |
|
jihpringle joined #evergreen |
| 12:10 |
Bmagic |
Dyrcona: I'm troubleshooting the SFTP ingram sftp thing. I've successfully logged into their server using a GUI tool, but the funny thing is, their server is sending my GUI tool a banner message. Which, I think, is tricking the Evergreen code. We're able to push orders, but we can't receive. When attempting to scp manually via scp command, I get this: "exec request failed on channel 0" - which is their server attempting to use my tty ( |
| 12:10 |
Bmagic |
according to google) |
| 12:11 |
Bmagic |
did you deal with this during testing? |
| 12:11 |
berick |
hm, just deployed locally. i've only testing sending so far. |
| 12:11 |
berick |
*tested |
| 12:13 |
Bmagic |
bouncing this off of you: scp account_redacted ftp.ingramcontent.com:/outgoing/filename_redacted.EIN ./test.ein |
| 12:13 |
Bmagic |
does that look sound? |
| 12:14 |
Bmagic |
(it prompts for the password) - which I provide, then I get the "exec request failed on channel 0" error |
| 12:16 |
berick |
Bmagic: that fails for me too, but using "sftp" instead of "scp" works OK |
| 15:20 |
|
kworstell-isl joined #evergreen |
| 16:21 |
Stompro |
I think the fake ISBN's that B&T creates for DVDs are not conflicting with real codes. Looks like Romania and Argentina were assigned 630 and 631 in 2023. |
| 16:21 |
Stompro |
s/are not/are now/ |
| 16:36 |
Stompro |
But, a test system with ISBN::Data from 2021 also doesn't load them... so maybe something else is causing this. |
| 16:39 |
Dyrcona |
Fake ISBNs were problem never a good idea. |
| 16:40 |
Dyrcona |
s/problem/probably/ but somehow problem is more appropriate though is not an adverb. |
| 16:50 |
mmorgan |
probably a problem |
| 15:37 |
Bmagic |
I think* it was saving a bib record, which in-turn queried reporter.old_super_simple_record - which is what's written on the bug report |
| 15:37 |
eeevil |
there are certain things that JIT wants to do that we have already poured a lot of brain power into manually tuning |
| 15:37 |
Bmagic |
that makes sense |
| 15:38 |
eeevil |
it's not that EG is incompatible with PG's JIT, it's that (just like autovac and stats targets, and lots of other stuff, tbh) it requires config that is tuned to your particular size and shape of data |
| 15:39 |
eeevil |
a small EG instance may benefit a BUNCH from JIT, and reporting might as well (I've tested neither, but I have suspicions) |
| 15:40 |
sandbergja |
thanks, eeevil, that's helpful info to have |
| 15:40 |
eeevil |
but(!) unless someone has the tuits to learn the feature well, and analyze our queries in that context, I agree that we should recommend disabling JIT for now |
| 15:40 |
Bmagic |
eeevil++ |
| 15:41 |
eeevil |
I'd actually propose we investigate (as a community, as broadly as we can manage) cross-column stats targets before jumping into JIT |
| 15:41 |
Dyrcona |
I'm OK with that, but I don't think I've every encountered an issue caused by JIT. Pg 16 (even unoptimized) has pretty much always been faster than Pg 10 for me. BUT! I've not run it in produciton, just in testing on a production-sized database with a production-sized server. |
| 15:41 |
eeevil |
(not to derail the JIT-specific discussion, but as a query tuning tool I think that will pay off for more folks) |
| 15:42 |
eeevil |
Dyrcona: I haven't seen JIT-specific issues either, but I think we can be fairly sure that plan stability is /higher/ without JIT (until we learn to tune it well) |
| 15:43 |
Bmagic |
Disabling it fixed our problem on a test-then-production instance |
| 15:43 |
Bmagic |
pretty sure it manifested as a time-out when clicking on save in the MARC editor |
| 15:43 |
Dyrcona |
So, we'll recommend disabling JIT in the installation instructions, then. |
| 15:43 |
eeevil |
which is to say, yes, some folks will lose out on some optimizations they get "for free" with JIT on, but things are much less likely to go sideways in ways some have seen |
| 07:53 |
|
collum joined #evergreen |
| 07:58 |
|
BDorsey joined #evergreen |
| 08:36 |
|
Dyrcona joined #evergreen |
| 08:42 |
Dyrcona |
Guess, I'll undo my testing changes on the test database and try the fetcher with the production settings and grab the files from Ingram's server. Maybe there's something that my test changes are masking. |
| 08:43 |
Dyrcona |
I did add vsftp on my sftp host, and configured one account to get files with ftp and others with sftp and that did not seem to cause files to be downloaded more than once in my test environment. I thought that using ftp and sftp with basically the same account would do it, but it didn't. |
| 08:46 |
Dyrcona |
I have also verified that FTP is working. |
| 09:03 |
Dyrcona |
Feels like there should be a way to do this with pg_dump/pg_restore, but so far no dice. I think I'll try a data-only dump with --disable-triggers and a truncate on the tables after. |
| 09:11 |
Dyrcona |
pg_dump: warning: there are circular foreign-key constraints among these tables: |
| 10:12 |
berick |
been meaning to move in that direction myself |
| 10:13 |
Dyrcona |
OK. I guess we agree more than I thought yesterday. |
| 10:13 |
berick |
also considering keeping local copies of the plain files -- before they go into the DB -- as a secondary backup for debugging, etc. |
| 10:13 |
Dyrcona |
I still haven't been able to replicate what I saw in production on my test environment, so I'm trying to restore the original settings. I'm running the fetcher to get caught up. |
| 10:14 |
berick |
possibly a lighter way to say "we already have this" than querying the DB |
| 10:14 |
Dyrcona |
Yeah, that is worth exploring. |
| 10:15 |
Dyrcona |
Back to the issue, I thought it must have to do with 1 account having sftp in the host and the others having ftp with the same credentials and directory, but even when I did that on my test setup, it didn't pull the files twice. |
| 10:16 |
Dyrcona |
This other account says X of 1722 targets.... |
| 10:34 |
|
jvwoolf joined #evergreen |
| 10:45 |
Dyrcona |
Ok. I've got only the Ingram accounts set to active. I did a run to get caught up and grabbed 63 files. The second run grabbed 0. Now, I'm going to set 1 account to sftp in the host and see what happens. There are other accounts with the same credentials using FTP. This should replicate what we saw in production. |
| 10:48 |
|
sandbergja joined #evergreen |
| 10:54 |
Dyrcona |
Files retrieved: 275 |
| 10:55 |
Dyrcona |
Wow! I think it is the host part of the duplicate check in retrieve_core after all. Dunno why it didn't trigger on my local test server, though. |
| 10:56 |
Dyrcona |
berick: I think we should regexp_replace('^[a-z]+://', host, '') in the host check, or we just need to update all of the same accounts at once? |
| 10:57 |
Dyrcona |
or, maybe [^:]+ instead of [a-z]+? |
| 10:58 |
Dyrcona |
I'm running the same again to see what happens. I suspect it should grab 275 again unless a new file has gone up. |
| 11:45 |
Dyrcona |
csharp_: We had something local that was written in PHP. Now, we're using Metabase. |
| 11:45 |
Dyrcona |
It's not built into Evergreen, though you could add links. |
| 11:46 |
Dyrcona |
So, setting all of the accounts with the same username and password to use sftp resolves the multiple files retrieval, as I thought it might. |
| 11:48 |
Dyrcona |
Moral of the story: Don't just set 1 account to SFTP to "test" it. It's all or nothing. |
| 11:51 |
jeff |
and take steps to ensure that nothing tries to fetch files while you make the config changes to all the accounts in question? |
| 11:52 |
Dyrcona |
jeff: A SQL update should be fast enough and much faster than doing it in through the client. |
| 11:54 |
Dyrcona |
I may suggest a small path to O::A::Acq::EDI.pm to remove the scheme from the URL stored in host field of acq.edi_account. Doing a schemeless comparison would also resolve the issue. |
| 15:40 |
StomproJ |
Yes, for a number of years. |
| 15:40 |
Bmagic |
StomproJ++ |
| 15:40 |
Bmagic |
we gotta get this merged then |
| 15:41 |
StomproJ |
Works for SMS call number also, and for the SMS test message.... although there is a bug there I need to document. |
| 15:42 |
Bmagic |
I'm reading https://gitlab.com/LARL/evergreen-larl/-/commit/73b424779e2a07e351cc5cd78c8c2ee32b12fd6e |
| 15:43 |
StomproJ |
Testing is probably hard since you need a flowroute account setup with real money involved. Plus all the SMS Campaign registration stuff. |
| 15:44 |
Bmagic |
right, so, it'll never get merged.... how do we "trump" that? |
| 15:44 |
Dyrcona |
From what I understand, the campaign registration is pretty much required if you want your messages to get through, even with the commercial gateway. |
| 15:44 |
StomproJ |
Oh, I need to update it for the latest version of their API also, which improves the delivery report data. |
| 15:45 |
Dyrcona |
Bmagic: You've got the power.... |
| 15:45 |
Bmagic |
the branch needs documentation and probably a pg test and a perl test |
| 15:46 |
Dyrcona |
EDI is also hard to test. |
| 15:47 |
StomproJ |
I could probably provide api keys for someone to test... I'll have to think about how to do that. |
| 15:48 |
Bmagic |
StomproJ++ |
| 15:48 |
Dyrcona |
I wouldn't want anyone to get in trouble for sharing test API keys. |
| 15:48 |
Bmagic |
maybe we could create a trial account for testing |
| 15:49 |
StomproJ |
I'll reach out to Flowroute and see if they can provide trial accounts. |
| 15:50 |
Bmagic |
Flowroute might get an increase in business, which makes it feel "icky" to only support them, but we gotta start somewhere |
| 15:50 |
Dyrcona |
There's precedent with stripe. Someone with an account has to test it. |
| 15:51 |
Dyrcona |
Well, that's the code that there is, supplied by a community member. I don't see anyone else sharing anything. (Says the guy who's not a fan of commercial solutions in general, but in this case, what can you do?) |
| 15:55 |
Bmagic |
I prefer Evergreen to talk to a commercial SMS API, rather than generate XML and ship that to MessageBee (for example) |
| 15:56 |
Bmagic |
Shipping XML and maintaining a bunch of AT template toolkit's is more annoying IMO |
| 15:59 |
StomproJ |
I'm using a php script to just send them to my email now. You just tell flowroute your callback url. |
| 16:00 |
Dyrcona |
"Click here to spam the world" |
| 16:00 |
Bmagic |
What's the next step to get this Flowroute code merged? We have a bug, and the code, though the code isn't in the "normal" form/branch method? A signoff maybe? It could use the documentation pieces (release notes, and proper doc injections). I'm sure everyone would welcome such an addition, and the SMS fire has been burning for a long time now |
| 16:01 |
Dyrcona |
Bmagic: You just said it. It needs signoff and release notes minimum. Tests and documentation would be most welcome. |
| 16:02 |
Bmagic |
Seems pretty doable to get it over the finish line IMO. StomproJ: maybe we can divide the tasks? |
| 16:04 |
Bmagic |
I think it would be "easier" (for me) if it were in the working repo |
| 16:04 |
StomproJ |
Sure, I think just having someone else interested in it helps... Probably docs on setting up a flowroute account also... |
| 16:17 |
Bmagic |
sweet, StomproJ: you good with rolling with the collab branch? |
| 16:17 |
StomproJ |
Sure, sounds good. |
| 16:17 |
Dyrcona |
Bmagic++ StomproJ++ |
| 16:18 |
Bmagic |
Sounded like you were interested in the documentation additions, and release notes? I can try my hand a tthe tests |
| 16:18 |
StomproJ |
Here are all the flowroute related commits that I have in our custom production repo. https://gitlab.com/LARL/evergreen-larl/-/commits/rel_3_11_1-larl?search=flowroute |
| 16:19 |
Bmagic |
StomproJ: I think/hope those are the exact commits I just put onto our collab branch |
| 16:19 |
Dyrcona |
https://gitlab.com/LARL/evergreen-larl/-/commit/3d4cdbc7d6260640e5f086c2965dccc3797e27d1 isn't in the collab branch, and I wonder if it is necessary. |
| 13:32 |
berick |
Dyrcona: i'd stop short of deleting the files after download since that's new behavior, but fixing the dupe detection is obv. critical. |
| 13:33 |
Dyrcona |
In the long run, I think we should delete the files, or make it an option. As far as I can tell, Ingram isn't cleaning up the outgoing directory. I have not checked other vendors, yet. |
| 13:34 |
Dyrcona |
So, I'll agree with that not being a bug fix. |
| 13:36 |
Dyrcona |
Meanwhile, I can test with local servers and messages from production. |
| 13:36 |
Dyrcona |
Wonder if the fetcher works with the debugger? |
| 13:47 |
Dyrcona |
I swear that I saw the duplicate detection code last week, but can I find it right now? Nope. |
| 13:48 |
Dyrcona |
It's probably in OpenILS::Application::Acq::EDI->process_retrieval |
| 15:26 |
|
jvwoolf joined #evergreen |
| 16:03 |
|
sandbergja joined #evergreen |
| 16:11 |
|
jvwoolf joined #evergreen |
| 16:30 |
Dyrcona |
OK. I cannot reproduce the duplicate retrieval in any of my 3 test environments with recent files. |
| 16:31 |
Dyrcona |
This requires further investigation. |
| 16:52 |
Dyrcona |
Well, that's that fo today. |
| 16:59 |
|
Guest21 joined #evergreen |
| 09:55 |
Dyrcona |
I think you do have to specify the remote though when pushing a new branch, if you didn't already set it to track something. |
| 10:11 |
|
mmorgan joined #evergreen |
| 11:49 |
|
jihpringle joined #evergreen |
| 12:14 |
Dyrcona |
Odd. In production, the SFTP code seems to be setting the remote filename to "1" while on my test vm it set it to the actual remote filename. I've compared RemoteAccount.pm between production and the test system, and they are identical. |
| 12:14 |
Dyrcona |
Also both are on the same release of Ubuntu with the same Perl version, and the same Net::SFTP::Foreign module version. |
| 12:16 |
Dyrcona |
I wonder if it is possibly a difference in the server? Could the SFTP server affect the return value of $sftp->put()? |
| 12:21 |
Dyrcona |
Has anyone else started using/testing SFTP with Ingram in production? If so, have you noticed anything in the acq.edi_message table, notably remote_file having just a "1" in it? |
| 12:41 |
Dyrcona |
i think there's a bug in the put_sftp function that somehow escaped testing. I unfortunately do not have syslogs on the test machine going back far enough to see what happened there. I'm going to run another test. |
| 12:50 |
Dyrcona |
Actually, I think I see the bug on one of my test systems. I'm going to test again and see what happens. |
| 13:05 |
|
sleary joined #evergreen |
| 13:44 |
Dyrcona |
Bleh. None of the production data is loading today. Too much missing extra data: bibs, etc. |
| 13:46 |
Dyrcona |
When I tried to add items via brief record, I get a failure to update the database. As I said, I don't know what I'm doing in the acq interface. |
| 14:07 |
* Dyrcona |
reloads a database dump or two.... |
| 14:11 |
|
collum joined #evergreen |
| 14:12 |
Dyrcona |
I suppose I could just wait until next week, but I really want to verify this bug. |
| 14:16 |
Dyrcona |
Found it in 1 account on a database that I have not reloaded, yet! remote_file is 1 for one of the purchase orders that I successfully tested. |
| 14:20 |
Dyrcona |
So, I'm going to test on one of the vms with the patch applied to see if that fixes it. |
| 14:20 |
Dyrcona |
But, first, the database must finish reloading. |
| 14:32 |
Dyrcona |
Lp 2060153 |
| 14:32 |
pinesol |
Launchpad bug 2060153 in Evergreen 3.12 "EDI SFTP Records wrong remote file when sending purchase order " [Undecided,New] https://launchpad.net/bugs/2060153 |
| 14:45 |
Dyrcona |
audit_time has fractions of a second and order_date does not. |
| 14:52 |
|
stephengwills joined #evergreen |
| 14:58 |
jeff |
yeah, timestamps that are set by DEFAULT NOW() in the database will go backward in time when those rows are later updated by... just about everything else in Evergreen. |
| 15:00 |
Dyrcona |
It makes me wonder how I got any data to test last time, though. I should check the auditor cleanup for that table. |
| 15:15 |
|
sleary_ joined #evergreen |
| 15:18 |
Dyrcona |
Ah. I see. The purchase order was updated in produciton since I last grabbed data, so the timestamps lost some precision. |
| 15:18 |
Dyrcona |
s/produciton/production/ |
| 10:28 |
|
JBoyer joined #evergreen |
| 11:09 |
Bmagic |
JBoyer: Dyrcona: I have the branch staged locally (on main) - would you mind reviewing my git situation before* I push? https://pastebin.com/tsBk2LrG |
| 11:10 |
Bmagic |
I included the last commit on remote main at the bottom of that paste for context |
| 11:18 |
Bmagic |
I tested the branch as far as confirming that Evergreen will build after the patch is applied to main (and concerto installs fresh) |
| 11:28 |
Dyrcona |
Bmagic: It looks allright, but don't use those commit messages if you put that into main. ;) |
| 11:29 |
Dyrcona |
Bmagic: Do you have a way to test the EDI transfer? |
| 11:29 |
Bmagic |
I'm sure you noticed that the commit messages are the combined blocks from the squashed commits for each author |
| 11:29 |
Bmagic |
testing EDI transfer: no |
| 11:30 |
Dyrcona |
Yeah, I noticed. I don't mind the way you squashed them, but I'd edit the commit messages. |
| 11:30 |
Dyrcona |
I used another vm and modified a couple of EDI accounts. I had a 1,000 word essay on it but Lp rejected the comment. :) |
| 11:31 |
Bmagic |
Like changing the wording? Maybe you could provide your suggested changes in a pastebin? |
| 11:32 |
berick |
safe to assume Ingram already has SFTP running on their side? |
| 11:32 |
Dyrcona |
I'm in a local meeting where we earlier discussed setting it up in production next week and testing with 1 library that uses Ingram. |
| 11:34 |
Dyrcona |
You could test it locally with another user account on the same vm. |
| 11:34 |
Bmagic |
hmm, that's an interesting idea |
| 11:35 |
Bmagic |
I do have the code currently installed and propped up on a local machine... I think you're saying I could create a linux user on the same machine and stage a PO to post to that user's home folder via ssh |
| 11:40 |
berick |
ok, confirmed the existing Ingram FTP host also supports sftp connections |
| 13:09 |
|
mantis left #evergreen |
| 13:23 |
* Dyrcona |
is about to switch networks. |
| 13:25 |
|
Dyrcona joined #evergreen |
| 13:25 |
Bmagic |
I got it tested yall! Successfully sent EDI message to: sftp://localhost |
| 13:26 |
Dyrcona |
Bmagic++ |
| 13:26 |
Bmagic |
I'm going to remove the patch and make sure it doesn't* work without it |
| 13:26 |
Dyrcona |
The tricky part was getting it to pick up messages. |
| 13:27 |
Bmagic |
yeah, that'd require hand-crafting the response EDI |
| 13:27 |
Bmagic |
if it can push, it ought to be able to pull right? |
| 13:27 |
Dyrcona |
That's why I used production data on a test system that was a bit behind. |
| 13:28 |
Dyrcona |
You can pull the EDI out of acq.edi_message. |
| 13:29 |
Dyrcona |
Bmagic: The code to pull (get) is different and it didn't work until yesterday afternoon. |
| 13:31 |
Bmagic |
so the return EDI message can just be the same as the push EDI message? |
| 13:34 |
Dyrcona |
I seem to recall it not working for me years ago. csharp_ might be able to better explain the issues. |
| 13:37 |
Bmagic |
I'm verifiying that the patch is actually not* on my machine. I'm confirming that by looking at some of the code changes in the git commit(s) and seeing that they are not in the file in /usr/local/share/perl/5.34.0/OpenILS/Utils/RemoteAccount.pm and they aren't |
| 13:37 |
Bmagic |
so, I'll double check and restart Evergreen services and try again |
| 13:37 |
Dyrcona |
Sure. I'm not contradicting you. It may not work under some circumstances. Also, you should try to test getting a message off the remote. |
| 13:41 |
Dyrcona |
Full disclosure: I did not test with the pre-existing code this time around because a) I took the bug report at face value and b) I recall trying it years ago at MVLC on a test system and something didn't work right. (I don't remember the exact details.) |
| 13:44 |
Bmagic |
well, I'm pretty sure it pushes the EDI message without the patch, I'll check pull |
| 13:44 |
Dyrcona |
Bmagic: How is your EDI account setup? Do you have separate directories for incoming and outgoing files? |
| 13:45 |
Bmagic |
yeah |
| 14:05 |
Dyrcona |
True, but if it ain't broke, don't fix it. :) |
| 14:05 |
|
Guest2747 joined #evergreen |
| 14:07 |
Dyrcona |
It might be worth asking csharp__ to elaborate on the uses cases where the current code doesn't work. |
| 14:10 |
Dyrcona |
I also see something else that should be tweaked in the branch: StrictHostKeychecking ought to be StrictHostKeyChecking, but I have verified that the former works by testing from a vm with an account that never connected to the SFTP vm before. |
| 14:11 |
* Dyrcona |
wonders if that might have been the main issue all along, not having the remote host key in ~opensrf/.ssh/known_hosts. |
| 14:14 |
Bmagic |
hmmm |
| 14:17 |
mmorgan |
Bmagic: Dyrcona: This is from an Ingram service alert message "In October, we alerted you that as of April 15, 2024 we would no longer support non-secured FTP connections and that all existing and future FTP accounts would have to use Secure FTP (sFTP)." |
| 14:18 |
Dyrcona |
mmorgan: Yeah, we're aware of the deadline. Right now, the question is does the current implementation of SFTP in OpenILS/Utils/RemoteAccount.pm need to be replaced. |
| 14:18 |
mmorgan |
Ok, gotcha. |
| 14:18 |
Dyrcona |
I'm going to revert one of my two test systems and try what was in 3.7.4. |
| 14:27 |
Dyrcona |
I found an invoice I can dump from production and use as a test. |
| 14:32 |
Dyrcona |
Bmagic: I just did a test with an invoice and the current implementation did not pick it up. I removed the host key from ~opensrf/.ssh/known_hosts. I'll add it and see what happens. |
| 14:34 |
Dyrcona |
Nope. still says: Files retrieved: 0 |
| 14:38 |
Dyrcona |
Bmagic: On 3.7.4 on my test set up, the stock Evergreen RemoteAccount.pm failed to retrieve the file. With the Lp 2040514 branch applied, it worked, even without the key in ~opensrf/.ssh/known_hosts. |
| 14:38 |
pinesol |
Launchpad bug 2040514 in Evergreen 3.12 "EDI SFTP doesn't work" [High,Confirmed] https://launchpad.net/bugs/2040514 |
| 14:39 |
Bmagic |
ok then, I must have a bad test |
| 14:39 |
Dyrcona |
You might just have conditions where the current code works. Also, it could be more broken in older Evergreen. I'm going to try the same test again on main. |
| 14:40 |
Bmagic |
I'm using main as my control test |
| 14:40 |
* Dyrcona |
has another vm almost all set to go. |
| 14:41 |
Dyrcona |
All you have to do to test one implementation versus the other is swap out /usr/local/share/perl/5.34.0/OpenILS/Utils/RemoteAccount.pm. The 5.34.0 part will be different depending on your Perl version. |
| 14:50 |
Dyrcona |
Looks like the current implementation in main is working. I wonder what changed (if anything) and when? I wonder if it has something to do with Linux release and/or Perl module versions? |
| 14:50 |
Bmagic |
oh? |
| 14:51 |
Bmagic |
so, I don't have a broken test? |
| 14:51 |
Dyrcona |
My main test vm is Ubuntu 22.04 and the other is Ubuntu 20.04. |
| 14:51 |
Bmagic |
yep, 22.04 and 5.34.0 perl for my testing |
| 14:51 |
Bmagic |
current main is both pushing and pulling via sftp://localhost (sans patch) |
| 14:51 |
Dyrcona |
It did not work for me on 20.04 with Perl 5.30. |
| 14:52 |
Bmagic |
interesting |
| 14:52 |
Dyrcona |
I only tested pull. |
| 14:52 |
Dyrcona |
I'm going to test pull again on 20.04. I've added another file. |
| 14:54 |
Dyrcona |
Yeah, stock implementation definitely not working there with Evergreen 3.7.4. I don't want to upgrade that one to main right now. |
| 14:55 |
Dyrcona |
I wonder if it works on Bullseye or Bookworm? |
| 14:57 |
Bmagic |
if the patch makes it work for more/various installations of Evergreen on different platforms, then, I think we should merge |
| 09:09 |
|
jvwoolf joined #evergreen |
| 09:12 |
Rogan |
kmlussier morning |
| 09:12 |
Rogan |
mmorgan morning to you too! |
| 09:15 |
Dyrcona |
What I'm doing to copy acq data for testing the SFTP stuff works with a branch updated to main, but on 3.7.4 the legacy acq po view becomes unresponsive. |
| 09:21 |
Dyrcona |
Maybe acq is just unresponsive on this test system? |
| 09:23 |
Dyrcona |
This po has 17 lineitems. It shows 1, then after a minute or so I get the "this page is unresponsive" dialog. |
| 09:23 |
Dyrcona |
It would be nice if the dev tools would open. It's that unresponsive. |
| 09:25 |
Dyrcona |
Got the console open and nada. |
| 09:53 |
|
sleary joined #evergreen |
| 10:02 |
|
jvwoolf joined #evergreen |
| 12:05 |
|
jihpringle joined #evergreen |
| 12:07 |
Dyrcona |
So, I'm doing the restore again because my test data load blew up. |
| 13:32 |
|
jvwoolf joined #evergreen |
| 13:52 |
Dyrcona |
Nope. Acquisitions is still busted. I think this db/vm combinations just can't deal with it. |
| 13:56 |
Dyrcona |
"GET /js/dojo/openils/acq/nls/en-us/acq.js HTTP/1.0" 404 That's no big deal, right? That's just looking for the translated files, isn't it? |
| 14:00 |
Dyrcona |
ha! Rickie Lee Jones sings, "It's more trouble than it's worth." |
| 14:02 |
Dyrcona |
I also tried autogen.sh because I think org units might have changed after the restore. |
| 14:05 |
Dyrcona |
Ok. Now, I sign in with a different workstation, and it takes me straight to workstation registration. If I try to add this workstation, it says the workstation already exists. It shows me with no working location in the green bar. |
| 14:10 |
JBoyer |
Sometimes I still need to wipe out the browser's localstorage when a test system has been rebuilt, especially if the workstation names and ids no longer match up. |
| 14:10 |
Dyrcona |
jBoyre I'll try that. I have disable cache with dev tools turned, but that's not local storage, is it? :) |
| 14:13 |
Dyrcona |
JBoyer++ That fixed it. |
| 14:13 |
JBoyer |
\o/ |
| 14:15 |
Dyrcona |
Behavior looks the same so far. Should it load more than 1 lineitem when there are 17? |
| 14:18 |
Dyrcona |
acq is just busted on this setup, and I don't know why. The logs don't show any real errors. |
| 14:26 |
Dyrcona |
So, another pending purchase order with 0 lineitems from a year ago seems to load OK. The data that I copied from production is probably not quite right or somthing. |
| 14:29 |
Dyrcona |
maybe I can still test the fetcher with an "on order" po? If I can find one that has more messages in production than in the test database. |
| 14:30 |
Dyrcona |
I get the page is not responding with an existing on order po, so it's the data that I'm adding. |
| 14:39 |
kmlussier |
When poking around on the wiki this morning, I noticed that we must have made a decision back in 2017 to call monthly meetings held in this channel "development meetings" instead of "developer meetings." See gmcharlt_'s note at the top of the revision page here - https://wiki.evergreen-ils.org/doku.php?do=revisions&first=99&id=dev%3Ameetings. I'm going to update the agenda template to say Development Meetings too. I just wanted to men |
| 14:39 |
kmlussier |
tion it in case anyone notices the change and wonders why it was made. |
| 14:41 |
Dyrcona |
The Dojo stuff can't go away too soon for me. |
| 14:41 |
Dyrcona |
kmlussier++ |
| 14:41 |
kmlussier |
Dyrcona: Agreed. |
| 14:42 |
Dyrcona |
JBoyer: You might be pleased to know that retrieved an EPO via SFTP on my test server again today, despite my issues with the legacy acq interface. |
| 14:43 |
Dyrcona |
I suppose I'll try the invoice for this po, too. |
| 14:43 |
kmlussier |
We're all pleased to know that, Dyrcona. :) |
| 14:43 |
kmlussier |
Dyrcona++ |
| 09:35 |
Dyrcona |
Don't try this in production, kids: evergreen=# select cwmars.delete_acq_purchase_order(id) from acq.purchase_order; |
| 09:35 |
JBoyer |
FYI, if you haven't seen it yet there's an angular rewrite of the reporter on butternut.evergreencatalog.com (and a reports security improvement). Would be great to see one or both of these land in 3.13! |
| 09:39 |
Dyrcona |
Maybe I should have added " where state = 'pending'" It's taking a while. |
| 09:45 |
Dyrcona |
Is there a good way to disable EDI accounts in acq? Doesn't look like there's an active flag. I want to test EDI with SFTP on just a couple of accounts. |
| 09:46 |
Dyrcona |
Hah! I just realized that I'm running that function above in the wrong database. It's running "evergreen" and I was setting this up in "jasontest." |
| 09:47 |
abneiman |
Bmagic: can you please see if docs build broke again? I'm looking for things I put in earlier this week & I see the files in git but I'm not seeing the updates on the webpage(s) |
| 09:48 |
abneiman |
alternatively, pls let me know if I broke docs, lol |
| 11:10 |
pinesol |
Launchpad bug 1525394 in Evergreen "SIP patron part level holds respond blank" [Undecided,Confirmed] https://launchpad.net/bugs/1525394 - Assigned to Chris Sharp (chrissharp123) |
| 11:29 |
csharp_ |
berick: sick with a cold - feel free to grab it |
| 11:30 |
berick |
csharp_: ugh! |
| 11:30 |
csharp_ |
berick: I was able to test parts holds and the fix works, but I was never able to figure out how one places an issuance hold |
| 11:30 |
berick |
ohh, that's the one you were looking at before.. thanks for the info |
| 11:30 |
berick |
i'll see what I can do |
| 11:30 |
csharp_ |
thanks |
| 12:35 |
|
dickreckard joined #evergreen |
| 13:01 |
Dyrcona |
Well, I simplified things by getting my data transfer to work with \copy, so I don't have to copy the files to the database server. I am still working on foreign key constraints, though. Every time I add a new table, I come up with another fkey relationship that I have to add to the queries. |
| 13:01 |
Dyrcona |
Acq is complicated. |
| 13:04 |
Dyrcona |
Oh! That acq.purchase_order delete finished. It actually succeeded in deleting all of the purchase orders in that test database. |
| 13:32 |
pinesol |
News from commits: LP1525394 moving toward fixing SIP hold titles <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=7775c5e1dcd74c354f249849c863d774f339b5d9> |
| 13:32 |
pinesol |
News from commits: LP1525394: Return Titles for All Hold Types in SIP <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=ffec39a79bd09ba5ba7223f13c455d56e0ec277b> |
| 13:32 |
pinesol |
News from commits: LP1525394 SIP patron part level holds respond blank <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=91a5ea4345d7822169688ab85c4037f817887dfe> |
| 15:56 |
berick |
e.g. https://bugsquash2.mobiusconsortium.org/eg2/en-US/staff/acq/po/9 |
| 15:57 |
berick |
btw there's enough seed data in EG to create a new PO, add a brief record, add one copy, add a price, then activate |
| 15:59 |
Dyrcona |
berick++ |
| 15:59 |
berick |
well, more configs needed for actually testing the sftp part |
| 15:59 |
berick |
edi configs, etc. |
| 15:59 |
Dyrcona |
I'm using production data copied to an older produciton database. I had to check "Allow Activation with Zero-Item Line Items?" |
| 16:00 |
Dyrcona |
Yeah, I've switched the host fields for the provider/edi accounts that I'm testing. |
| 16:01 |
Dyrcona |
Are items actually created in the asset.copy table? Should l look for those and pull them over? I notice that there are links to the biblio.record_entry table. |
| 16:04 |
Dyrcona |
Yeah, I guess they are. I should pull that data over too, I suppose. |
| 16:05 |
Dyrcona |
I've been trying to get the data from the auditor tables before the purchase order was activated, so I suppose I should check the auditor tables in the case of a po that has been received? |
| 16:08 |
berick |
and you won't need "Allow Activation with Zero-.." if you just add a copy to the order |
| 16:16 |
Dyrcona |
berick: Uh-huh. Considering I'm happier in the database than the interface, I disagree with you. :) |
| 16:16 |
berick |
hah! gotcha |
| 16:19 |
Dyrcona |
I've got a test system with production data upgraded to main as of a week ago or so. The data is two weeks or more old, so I'd like to test with that. I've got another machine with an account set up to use for SFTP. I reckon I can generate POs and send them there, then grab the ORDRSP message from production, throw that file over on the SFTP vm and see if the fetcher can get it. |
| 16:26 |
Dyrcona |
Looks like i should also grab lineitem_detail, asset.copy and asset.call_number. |
| 16:31 |
|
jvwoolf joined #evergreen |
| 16:34 |
Dyrcona |
And, I need to grab acq.fund_debit as well. |
| 16:41 |
Dyrcona |
i suppose I can set the fund and fund_debit to null in lineitem_detail... I'll give that a try. |
| 16:43 |
Dyrcona |
berick: Should pull the asset.copy and call number through lineitem_detail, or should I null those fields, too? (I guess I could plumb the code to find out.) |
| 16:49 |
Dyrcona |
Ooh. I activated the order... |
| 16:49 |
Dyrcona |
Guess I have time to run the order push to see if the message goes to my test vm. |
| 16:53 |
Dyrcona |
Nope. no file got sent over. |
| 16:56 |
Dyrcona |
Right the edi_message was created but remote file is blank. I guess i'll pick this up on Monday. |
| 16:56 |
Dyrcona |
Have a great weekend, everyone! |
| 07:31 |
|
collum joined #evergreen |
| 07:59 |
|
BDorsey joined #evergreen |
| 08:23 |
JBoyer |
dickreckard, I've helped with some upgrades that skipped about that many versions, there are only a couple places that things could get weird or take a long time, |
| 08:23 |
JBoyer |
and if you setup a test database (you don't even need the newer version of Evergreen running to test the upgrade scripts) you should be able to plan for all of those. |
| 08:24 |
JBoyer |
And since all of the upgrade scripts either work or don't you don't have to worry about the db being left in a hard-to-debug state. |
| 08:28 |
|
Dyrcona joined #evergreen |
| 08:31 |
|
mantis joined #evergreen |
| 09:19 |
Bmagic |
dickreckard: I would go with the sql script upgrade approach. It would be much less time consuming |
| 09:20 |
JBoyer |
Well, for example, if you run the 3.2.2-3.2.3-upgrade-db.sql script, it will either work entirely leaving your db as expected for a 3.2.3 system, or it will fail and tell you what went wrong so you can fix the issue and then run the script again. |
| 09:20 |
JBoyer |
And if an upgrade fails, it rolls back every change it tried to make, so you'd be left with the same 3.2.2 db that you had before trying to apply the update. |
| 09:22 |
Bmagic |
JBoyer speaks truth! And I'll echo him: we always practice the upgrade SQL scripts on a test copy of the database in order to see which ones fail and why. Make the required adjustments to the SQL scripts to fit our database needs. Most* of the time we don't have to make changes. You won't know until you try |
| 09:23 |
|
dguarrac joined #evergreen |
| 09:30 |
Dyrcona |
dickreckard: Do you have access to another PostgreSQL database where you can test the database upgrade? |
| 09:33 |
|
smayo joined #evergreen |
| 09:33 |
* Bmagic |
waves at smayo |
| 09:34 |
|
jvwoolf joined #evergreen |
| 09:36 |
jvwoolf |
Bmagic++ |
| 09:36 |
jvwoolf |
#1 hype man |
| 09:37 |
Bmagic |
Can I get an Eeeeee! |
| 09:37 |
Dyrcona |
So, fun with autorenewals on a test system: 1 failed to renew last night because the circ drone couldn't get a transaction. Nothing in the Pg logs about it, except "there is not transaction in progress." |
| 09:37 |
smayo |
Eeeeee |
| 09:37 |
Dyrcona |
Very next autorenew event gets an error state, but from looking at the database and the logs, it actually succeeded. Both process by the same trigger drone. |
| 09:38 |
jvwoolf |
https://www.youtube.com/watch?v=0ipbu4msLSw |
| 09:38 |
dickreckard |
Bmagic: Dyrcona JBoyer, yes will go with the test db! thank you all for help :) |
| 09:38 |
Dyrcona |
No reason at all for the second to have the state='error'. |
| 09:39 |
Bmagic |
jvwoolf++ # love that battle cry |
| 09:39 |
Dyrcona |
dicreckard: I recommend you try this: https://gist.github.com/Dyrcona/00bd6b6290b6fbbb579c7f93b360ab0d It will create a database upgrade script between two arbitrary Evergreen versions. It requires git, and I guarantee the script will not work on the first try. |
| 09:56 |
Dyrcona |
Both events have the same update_time: 2024-03-21 05:43:13.394412-04. I think that might be significant. That they're being updated at the same time. |
| 10:04 |
Bmagic |
terranm++ # lol, good to know that yall know what I would type so you know when to be suspicious hahaha |
| 10:05 |
terranm |
Bmagic++ |
| 10:10 |
Bmagic |
csharp_: https://bugsquash3.mobiusconsortium.org is up and running at your service. I did login and test placing a hold and it worked! bugsquash2 has several patches that are candidates for having broke holds. I'll figure out which one and post on the bug |
| 10:15 |
|
kworstell_isl joined #evergreen |
| 10:16 |
|
kworstell_isl_ joined #evergreen |
| 10:24 |
Dyrcona |
ooh. got a real dead lock in a test system on Sunday: Process 3401997 ... blocked by process 3402007. ...Process 3402007 ... blocked by process 3401997. |
| 10:26 |
Dyrcona |
Looks like the hold targeter blocked a process trying to reindex the action.hold_request table and vice versa. |
| 10:26 |
* Dyrcona |
changes the hold targeter schedule. |
| 10:27 |
|
kworstell-isl joined #evergreen |
| 10:48 |
|
sleary joined #evergreen |
| 10:49 |
Rogan |
dickreckard I'm late to the party but echo jboyer's statement, just exporting isolated tables and reimporting them will be a bigger pain than the sql upgrades and you will lose a lot of data |
| 11:01 |
Dyrcona |
pg_dump/pg_restore usually does not work if the schema has changed. |
| 11:02 |
kenstir |
Can I get a clue how to set up my test server for Stripe payments? I have a stripe account, and I used the library settings editor to add things to BR1, but the BR1 patron with a fine does not see any "pay fines" button. Screenshot: https://launchpadlibrarian.net/720500896/2024-03-20-Library-Settings-Editor.png |
| 11:04 |
Dyrcona |
kenstir: You added your test settings for stripe and enabled stripe as the cc processor? If so, you should be able to "pay" with one of the stripe test cards. See the strip documentation. |
| 11:05 |
kenstir |
Dyrcona Yes I did but the issue is I don't see any pay button in EG when logged in as the patron. I do see the fine. |
| 11:06 |
Dyrcona |
you restarted services and apache2 after making the settings changes? they don't always have an immediate effect. |
| 11:06 |
mmorgan |
kenstir: Did you set the library setting Allow Credit Card Payments (credit.payments.allow) to TRUE also? |
| 14:10 |
berick |
Dyrcona: yeah, and I'll probably be poking at https://opensearch.org/ before too long as an Elastic replacement. |
| 14:12 |
jeffdavis |
Are we ok to officially recommend disabling JIT in Postgres? I don't recall if anyone had reservations about turning it off. (bug 2042158) |
| 14:12 |
pinesol |
Launchpad bug 2042158 in Evergreen "Postgres 12+ needs to have JIT disabled" [Undecided,Confirmed] https://launchpad.net/bugs/2042158 - Assigned to Galen Charlton (gmc) |
| 14:18 |
Dyrcona |
jeffdavis: I have reservations about turning it off, but I have not made them public. If we have queries that cause issues with the JIT we should probably fix the queries. I have not turned off JIT in my Pg 15 and Pg 16 test databases and I've not encountered the issue described in the bug. All of that said, don't let just me stop anything from going forward. |
| 14:23 |
Dyrcona |
Speaking of PostgreSQL settings, this is mainly for Bmagic: I think I'm going back to 1.1 for random_page_cost. A couple of queries that I've run since setting it to 1.0 seem slower. |
| 14:25 |
Dyrcona |
... on Pg10 anyway. |
| 14:40 |
JBoyer |
I haven't weighed in much on that or had much time to do much lately but feel the same as Dyrcona re: turning off the JIT unconditionally. |
