| 09:55 |
Dyrcona |
I think you do have to specify the remote though when pushing a new branch, if you didn't already set it to track something. |
| 10:11 |
|
mmorgan joined #evergreen |
| 11:49 |
|
jihpringle joined #evergreen |
| 12:14 |
Dyrcona |
Odd. In production, the SFTP code seems to be setting the remote filename to "1" while on my test vm it set it to the actual remote filename. I've compared RemoteAccount.pm between production and the test system, and they are identical. |
| 12:14 |
Dyrcona |
Also both are on the same release of Ubuntu with the same Perl version, and the same Net::SFTP::Foreign module version. |
| 12:16 |
Dyrcona |
I wonder if it is possibly a difference in the server? Could the SFTP server affect the return value of $sftp->put()? |
| 12:21 |
Dyrcona |
Has anyone else started using/testing SFTP with Ingram in production? If so, have you noticed anything in the acq.edi_message table, notably remote_file having just a "1" in it? |
| 12:41 |
Dyrcona |
i think there's a bug in the put_sftp function that somehow escaped testing. I unfortunately do not have syslogs on the test machine going back far enough to see what happened there. I'm going to run another test. |
| 12:50 |
Dyrcona |
Actually, I think I see the bug on one of my test systems. I'm going to test again and see what happens. |
| 13:05 |
|
sleary joined #evergreen |
| 13:44 |
Dyrcona |
Bleh. None of the production data is loading today. Too much missing extra data: bibs, etc. |
| 13:46 |
Dyrcona |
When I tried to add items via brief record, I get a failure to update the database. As I said, I don't know what I'm doing in the acq interface. |
| 14:07 |
* Dyrcona |
reloads a database dump or two.... |
| 14:11 |
|
collum joined #evergreen |
| 14:12 |
Dyrcona |
I suppose I could just wait until next week, but I really want to verify this bug. |
| 14:16 |
Dyrcona |
Found it in 1 account on a database that I have not reloaded, yet! remote_file is 1 for one of the purchase orders that I successfully tested. |
| 14:20 |
Dyrcona |
So, I'm going to test on one of the vms with the patch applied to see if that fixes it. |
| 14:20 |
Dyrcona |
But, first, the database must finish reloading. |
| 14:32 |
Dyrcona |
Lp 2060153 |
| 14:32 |
pinesol |
Launchpad bug 2060153 in Evergreen 3.12 "EDI SFTP Records wrong remote file when sending purchase order " [Undecided,New] https://launchpad.net/bugs/2060153 |
| 10:28 |
|
JBoyer joined #evergreen |
| 11:09 |
Bmagic |
JBoyer: Dyrcona: I have the branch staged locally (on main) - would you mind reviewing my git situation before* I push? https://pastebin.com/tsBk2LrG |
| 11:10 |
Bmagic |
I included the last commit on remote main at the bottom of that paste for context |
| 11:18 |
Bmagic |
I tested the branch as far as confirming that Evergreen will build after the patch is applied to main (and concerto installs fresh) |
| 11:28 |
Dyrcona |
Bmagic: It looks allright, but don't use those commit messages if you put that into main. ;) |
| 11:29 |
Dyrcona |
Bmagic: Do you have a way to test the EDI transfer? |
| 11:29 |
Bmagic |
I'm sure you noticed that the commit messages are the combined blocks from the squashed commits for each author |
| 11:29 |
Bmagic |
testing EDI transfer: no |
| 11:30 |
Dyrcona |
Yeah, I noticed. I don't mind the way you squashed them, but I'd edit the commit messages. |
| 11:30 |
Dyrcona |
I used another vm and modified a couple of EDI accounts. I had a 1,000 word essay on it but Lp rejected the comment. :) |
| 11:31 |
Bmagic |
Like changing the wording? Maybe you could provide your suggested changes in a pastebin? |
| 11:32 |
berick |
safe to assume Ingram already has SFTP running on their side? |
| 11:32 |
Dyrcona |
I'm in a local meeting where we earlier discussed setting it up in production next week and testing with 1 library that uses Ingram. |
| 11:34 |
Dyrcona |
You could test it locally with another user account on the same vm. |
| 11:34 |
Bmagic |
hmm, that's an interesting idea |
| 11:35 |
Bmagic |
I do have the code currently installed and propped up on a local machine... I think you're saying I could create a linux user on the same machine and stage a PO to post to that user's home folder via ssh |
| 11:40 |
berick |
ok, confirmed the existing Ingram FTP host also supports sftp connections |
| 13:09 |
|
mantis left #evergreen |
| 13:23 |
* Dyrcona |
is about to switch networks. |
| 13:25 |
|
Dyrcona joined #evergreen |
| 13:25 |
Bmagic |
I got it tested yall! Successfully sent EDI message to: sftp://localhost |
| 13:26 |
Dyrcona |
Bmagic++ |
| 13:26 |
Bmagic |
I'm going to remove the patch and make sure it doesn't* work without it |
| 13:26 |
Dyrcona |
The tricky part was getting it to pick up messages. |
| 13:27 |
Bmagic |
yeah, that'd require hand-crafting the response EDI |
| 13:27 |
Bmagic |
if it can push, it ought to be able to pull right? |
| 13:27 |
Dyrcona |
That's why I used production data on a test system that was a bit behind. |
| 13:28 |
Dyrcona |
You can pull the EDI out of acq.edi_message. |
| 13:29 |
Dyrcona |
Bmagic: The code to pull (get) is different and it didn't work until yesterday afternoon. |
| 13:31 |
Bmagic |
so the return EDI message can just be the same as the push EDI message? |
| 13:34 |
Dyrcona |
I seem to recall it not working for me years ago. csharp_ might be able to better explain the issues. |
| 13:37 |
Bmagic |
I'm verifiying that the patch is actually not* on my machine. I'm confirming that by looking at some of the code changes in the git commit(s) and seeing that they are not in the file in /usr/local/share/perl/5.34.0/OpenILS/Utils/RemoteAccount.pm and they aren't |
| 13:37 |
Bmagic |
so, I'll double check and restart Evergreen services and try again |
| 13:37 |
Dyrcona |
Sure. I'm not contradicting you. It may not work under some circumstances. Also, you should try to test getting a message off the remote. |
| 13:41 |
Dyrcona |
Full disclosure: I did not test with the pre-existing code this time around because a) I took the bug report at face value and b) I recall trying it years ago at MVLC on a test system and something didn't work right. (I don't remember the exact details.) |
| 13:44 |
Bmagic |
well, I'm pretty sure it pushes the EDI message without the patch, I'll check pull |
| 13:44 |
Dyrcona |
Bmagic: How is your EDI account setup? Do you have separate directories for incoming and outgoing files? |
| 13:45 |
Bmagic |
yeah |
| 14:05 |
Dyrcona |
True, but if it ain't broke, don't fix it. :) |
| 14:05 |
|
Guest2747 joined #evergreen |
| 14:07 |
Dyrcona |
It might be worth asking csharp__ to elaborate on the uses cases where the current code doesn't work. |
| 14:10 |
Dyrcona |
I also see something else that should be tweaked in the branch: StrictHostKeychecking ought to be StrictHostKeyChecking, but I have verified that the former works by testing from a vm with an account that never connected to the SFTP vm before. |
| 14:11 |
* Dyrcona |
wonders if that might have been the main issue all along, not having the remote host key in ~opensrf/.ssh/known_hosts. |
| 14:14 |
Bmagic |
hmmm |
| 14:17 |
mmorgan |
Bmagic: Dyrcona: This is from an Ingram service alert message "In October, we alerted you that as of April 15, 2024 we would no longer support non-secured FTP connections and that all existing and future FTP accounts would have to use Secure FTP (sFTP)." |
| 14:18 |
Dyrcona |
mmorgan: Yeah, we're aware of the deadline. Right now, the question is does the current implementation of SFTP in OpenILS/Utils/RemoteAccount.pm need to be replaced. |
| 14:18 |
mmorgan |
Ok, gotcha. |
| 14:18 |
Dyrcona |
I'm going to revert one of my two test systems and try what was in 3.7.4. |
| 14:27 |
Dyrcona |
I found an invoice I can dump from production and use as a test. |
| 14:32 |
Dyrcona |
Bmagic: I just did a test with an invoice and the current implementation did not pick it up. I removed the host key from ~opensrf/.ssh/known_hosts. I'll add it and see what happens. |
| 14:34 |
Dyrcona |
Nope. still says: Files retrieved: 0 |
| 14:38 |
Dyrcona |
Bmagic: On 3.7.4 on my test set up, the stock Evergreen RemoteAccount.pm failed to retrieve the file. With the Lp 2040514 branch applied, it worked, even without the key in ~opensrf/.ssh/known_hosts. |
| 14:38 |
pinesol |
Launchpad bug 2040514 in Evergreen 3.12 "EDI SFTP doesn't work" [High,Confirmed] https://launchpad.net/bugs/2040514 |
| 14:39 |
Bmagic |
ok then, I must have a bad test |
| 14:39 |
Dyrcona |
You might just have conditions where the current code works. Also, it could be more broken in older Evergreen. I'm going to try the same test again on main. |
| 14:40 |
Bmagic |
I'm using main as my control test |
| 14:40 |
* Dyrcona |
has another vm almost all set to go. |
| 14:41 |
Dyrcona |
All you have to do to test one implementation versus the other is swap out /usr/local/share/perl/5.34.0/OpenILS/Utils/RemoteAccount.pm. The 5.34.0 part will be different depending on your Perl version. |
| 14:50 |
Dyrcona |
Looks like the current implementation in main is working. I wonder what changed (if anything) and when? I wonder if it has something to do with Linux release and/or Perl module versions? |
| 14:50 |
Bmagic |
oh? |
| 14:51 |
Bmagic |
so, I don't have a broken test? |
| 14:51 |
Dyrcona |
My main test vm is Ubuntu 22.04 and the other is Ubuntu 20.04. |
| 14:51 |
Bmagic |
yep, 22.04 and 5.34.0 perl for my testing |
| 14:51 |
Bmagic |
current main is both pushing and pulling via sftp://localhost (sans patch) |
| 14:51 |
Dyrcona |
It did not work for me on 20.04 with Perl 5.30. |
| 14:52 |
Bmagic |
interesting |
| 14:52 |
Dyrcona |
I only tested pull. |
| 14:52 |
Dyrcona |
I'm going to test pull again on 20.04. I've added another file. |
| 14:54 |
Dyrcona |
Yeah, stock implementation definitely not working there with Evergreen 3.7.4. I don't want to upgrade that one to main right now. |
| 14:55 |
Dyrcona |
I wonder if it works on Bullseye or Bookworm? |
| 14:57 |
Bmagic |
if the patch makes it work for more/various installations of Evergreen on different platforms, then, I think we should merge |
| 09:09 |
|
jvwoolf joined #evergreen |
| 09:12 |
Rogan |
kmlussier morning |
| 09:12 |
Rogan |
mmorgan morning to you too! |
| 09:15 |
Dyrcona |
What I'm doing to copy acq data for testing the SFTP stuff works with a branch updated to main, but on 3.7.4 the legacy acq po view becomes unresponsive. |
| 09:21 |
Dyrcona |
Maybe acq is just unresponsive on this test system? |
| 09:23 |
Dyrcona |
This po has 17 lineitems. It shows 1, then after a minute or so I get the "this page is unresponsive" dialog. |
| 09:23 |
Dyrcona |
It would be nice if the dev tools would open. It's that unresponsive. |
| 09:25 |
Dyrcona |
Got the console open and nada. |
| 09:53 |
|
sleary joined #evergreen |
| 10:02 |
|
jvwoolf joined #evergreen |
| 12:05 |
|
jihpringle joined #evergreen |
| 12:07 |
Dyrcona |
So, I'm doing the restore again because my test data load blew up. |
| 13:32 |
|
jvwoolf joined #evergreen |
| 13:52 |
Dyrcona |
Nope. Acquisitions is still busted. I think this db/vm combinations just can't deal with it. |
| 13:56 |
Dyrcona |
"GET /js/dojo/openils/acq/nls/en-us/acq.js HTTP/1.0" 404 That's no big deal, right? That's just looking for the translated files, isn't it? |
| 14:00 |
Dyrcona |
ha! Rickie Lee Jones sings, "It's more trouble than it's worth." |
| 14:02 |
Dyrcona |
I also tried autogen.sh because I think org units might have changed after the restore. |
| 14:05 |
Dyrcona |
Ok. Now, I sign in with a different workstation, and it takes me straight to workstation registration. If I try to add this workstation, it says the workstation already exists. It shows me with no working location in the green bar. |
| 14:10 |
JBoyer |
Sometimes I still need to wipe out the browser's localstorage when a test system has been rebuilt, especially if the workstation names and ids no longer match up. |
| 14:10 |
Dyrcona |
jBoyre I'll try that. I have disable cache with dev tools turned, but that's not local storage, is it? :) |
| 14:13 |
Dyrcona |
JBoyer++ That fixed it. |
| 14:13 |
JBoyer |
\o/ |
| 14:15 |
Dyrcona |
Behavior looks the same so far. Should it load more than 1 lineitem when there are 17? |
| 14:18 |
Dyrcona |
acq is just busted on this setup, and I don't know why. The logs don't show any real errors. |
| 14:26 |
Dyrcona |
So, another pending purchase order with 0 lineitems from a year ago seems to load OK. The data that I copied from production is probably not quite right or somthing. |
| 14:29 |
Dyrcona |
maybe I can still test the fetcher with an "on order" po? If I can find one that has more messages in production than in the test database. |
| 14:30 |
Dyrcona |
I get the page is not responding with an existing on order po, so it's the data that I'm adding. |
| 14:39 |
kmlussier |
When poking around on the wiki this morning, I noticed that we must have made a decision back in 2017 to call monthly meetings held in this channel "development meetings" instead of "developer meetings." See gmcharlt_'s note at the top of the revision page here - https://wiki.evergreen-ils.org/doku.php?do=revisions&first=99&id=dev%3Ameetings. I'm going to update the agenda template to say Development Meetings too. I just wanted to men |
| 14:39 |
kmlussier |
tion it in case anyone notices the change and wonders why it was made. |
| 14:41 |
Dyrcona |
The Dojo stuff can't go away too soon for me. |
| 14:41 |
Dyrcona |
kmlussier++ |
| 14:41 |
kmlussier |
Dyrcona: Agreed. |
| 14:42 |
Dyrcona |
JBoyer: You might be pleased to know that retrieved an EPO via SFTP on my test server again today, despite my issues with the legacy acq interface. |
| 14:43 |
Dyrcona |
I suppose I'll try the invoice for this po, too. |
| 14:43 |
kmlussier |
We're all pleased to know that, Dyrcona. :) |
| 14:43 |
kmlussier |
Dyrcona++ |
| 09:35 |
Dyrcona |
Don't try this in production, kids: evergreen=# select cwmars.delete_acq_purchase_order(id) from acq.purchase_order; |
| 09:35 |
JBoyer |
FYI, if you haven't seen it yet there's an angular rewrite of the reporter on butternut.evergreencatalog.com (and a reports security improvement). Would be great to see one or both of these land in 3.13! |
| 09:39 |
Dyrcona |
Maybe I should have added " where state = 'pending'" It's taking a while. |
| 09:45 |
Dyrcona |
Is there a good way to disable EDI accounts in acq? Doesn't look like there's an active flag. I want to test EDI with SFTP on just a couple of accounts. |
| 09:46 |
Dyrcona |
Hah! I just realized that I'm running that function above in the wrong database. It's running "evergreen" and I was setting this up in "jasontest." |
| 09:47 |
abneiman |
Bmagic: can you please see if docs build broke again? I'm looking for things I put in earlier this week & I see the files in git but I'm not seeing the updates on the webpage(s) |
| 09:48 |
abneiman |
alternatively, pls let me know if I broke docs, lol |
| 11:10 |
pinesol |
Launchpad bug 1525394 in Evergreen "SIP patron part level holds respond blank" [Undecided,Confirmed] https://launchpad.net/bugs/1525394 - Assigned to Chris Sharp (chrissharp123) |
| 11:29 |
csharp_ |
berick: sick with a cold - feel free to grab it |
| 11:30 |
berick |
csharp_: ugh! |
| 11:30 |
csharp_ |
berick: I was able to test parts holds and the fix works, but I was never able to figure out how one places an issuance hold |
| 11:30 |
berick |
ohh, that's the one you were looking at before.. thanks for the info |
| 11:30 |
berick |
i'll see what I can do |
| 11:30 |
csharp_ |
thanks |
| 12:35 |
|
dickreckard joined #evergreen |
| 13:01 |
Dyrcona |
Well, I simplified things by getting my data transfer to work with \copy, so I don't have to copy the files to the database server. I am still working on foreign key constraints, though. Every time I add a new table, I come up with another fkey relationship that I have to add to the queries. |
| 13:01 |
Dyrcona |
Acq is complicated. |
| 13:04 |
Dyrcona |
Oh! That acq.purchase_order delete finished. It actually succeeded in deleting all of the purchase orders in that test database. |
| 13:32 |
pinesol |
News from commits: LP1525394 moving toward fixing SIP hold titles <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=7775c5e1dcd74c354f249849c863d774f339b5d9> |
| 13:32 |
pinesol |
News from commits: LP1525394: Return Titles for All Hold Types in SIP <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=ffec39a79bd09ba5ba7223f13c455d56e0ec277b> |
| 13:32 |
pinesol |
News from commits: LP1525394 SIP patron part level holds respond blank <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=91a5ea4345d7822169688ab85c4037f817887dfe> |
| 15:56 |
berick |
e.g. https://bugsquash2.mobiusconsortium.org/eg2/en-US/staff/acq/po/9 |
| 15:57 |
berick |
btw there's enough seed data in EG to create a new PO, add a brief record, add one copy, add a price, then activate |
| 15:59 |
Dyrcona |
berick++ |
| 15:59 |
berick |
well, more configs needed for actually testing the sftp part |
| 15:59 |
berick |
edi configs, etc. |
| 15:59 |
Dyrcona |
I'm using production data copied to an older produciton database. I had to check "Allow Activation with Zero-Item Line Items?" |
| 16:00 |
Dyrcona |
Yeah, I've switched the host fields for the provider/edi accounts that I'm testing. |
| 16:01 |
Dyrcona |
Are items actually created in the asset.copy table? Should l look for those and pull them over? I notice that there are links to the biblio.record_entry table. |
| 16:04 |
Dyrcona |
Yeah, I guess they are. I should pull that data over too, I suppose. |
| 16:05 |
Dyrcona |
I've been trying to get the data from the auditor tables before the purchase order was activated, so I suppose I should check the auditor tables in the case of a po that has been received? |
| 16:08 |
berick |
and you won't need "Allow Activation with Zero-.." if you just add a copy to the order |
| 16:16 |
Dyrcona |
berick: Uh-huh. Considering I'm happier in the database than the interface, I disagree with you. :) |
| 16:16 |
berick |
hah! gotcha |
| 16:19 |
Dyrcona |
I've got a test system with production data upgraded to main as of a week ago or so. The data is two weeks or more old, so I'd like to test with that. I've got another machine with an account set up to use for SFTP. I reckon I can generate POs and send them there, then grab the ORDRSP message from production, throw that file over on the SFTP vm and see if the fetcher can get it. |
| 16:26 |
Dyrcona |
Looks like i should also grab lineitem_detail, asset.copy and asset.call_number. |
| 16:31 |
|
jvwoolf joined #evergreen |
| 16:34 |
Dyrcona |
And, I need to grab acq.fund_debit as well. |
| 16:41 |
Dyrcona |
i suppose I can set the fund and fund_debit to null in lineitem_detail... I'll give that a try. |
| 16:43 |
Dyrcona |
berick: Should pull the asset.copy and call number through lineitem_detail, or should I null those fields, too? (I guess I could plumb the code to find out.) |
| 16:49 |
Dyrcona |
Ooh. I activated the order... |
| 16:49 |
Dyrcona |
Guess I have time to run the order push to see if the message goes to my test vm. |
| 16:53 |
Dyrcona |
Nope. no file got sent over. |
| 16:56 |
Dyrcona |
Right the edi_message was created but remote file is blank. I guess i'll pick this up on Monday. |
| 16:56 |
Dyrcona |
Have a great weekend, everyone! |
| 07:31 |
|
collum joined #evergreen |
| 07:59 |
|
BDorsey joined #evergreen |
| 08:23 |
JBoyer |
dickreckard, I've helped with some upgrades that skipped about that many versions, there are only a couple places that things could get weird or take a long time, |
| 08:23 |
JBoyer |
and if you setup a test database (you don't even need the newer version of Evergreen running to test the upgrade scripts) you should be able to plan for all of those. |
| 08:24 |
JBoyer |
And since all of the upgrade scripts either work or don't you don't have to worry about the db being left in a hard-to-debug state. |
| 08:28 |
|
Dyrcona joined #evergreen |
| 08:31 |
|
mantis joined #evergreen |
| 09:19 |
Bmagic |
dickreckard: I would go with the sql script upgrade approach. It would be much less time consuming |
| 09:20 |
JBoyer |
Well, for example, if you run the 3.2.2-3.2.3-upgrade-db.sql script, it will either work entirely leaving your db as expected for a 3.2.3 system, or it will fail and tell you what went wrong so you can fix the issue and then run the script again. |
| 09:20 |
JBoyer |
And if an upgrade fails, it rolls back every change it tried to make, so you'd be left with the same 3.2.2 db that you had before trying to apply the update. |
| 09:22 |
Bmagic |
JBoyer speaks truth! And I'll echo him: we always practice the upgrade SQL scripts on a test copy of the database in order to see which ones fail and why. Make the required adjustments to the SQL scripts to fit our database needs. Most* of the time we don't have to make changes. You won't know until you try |
| 09:23 |
|
dguarrac joined #evergreen |
| 09:30 |
Dyrcona |
dickreckard: Do you have access to another PostgreSQL database where you can test the database upgrade? |
| 09:33 |
|
smayo joined #evergreen |
| 09:33 |
* Bmagic |
waves at smayo |
| 09:34 |
|
jvwoolf joined #evergreen |
| 09:36 |
jvwoolf |
Bmagic++ |
| 09:36 |
jvwoolf |
#1 hype man |
| 09:37 |
Bmagic |
Can I get an Eeeeee! |
| 09:37 |
Dyrcona |
So, fun with autorenewals on a test system: 1 failed to renew last night because the circ drone couldn't get a transaction. Nothing in the Pg logs about it, except "there is not transaction in progress." |
| 09:37 |
smayo |
Eeeeee |
| 09:37 |
Dyrcona |
Very next autorenew event gets an error state, but from looking at the database and the logs, it actually succeeded. Both process by the same trigger drone. |
| 09:38 |
jvwoolf |
https://www.youtube.com/watch?v=0ipbu4msLSw |
| 09:38 |
dickreckard |
Bmagic: Dyrcona JBoyer, yes will go with the test db! thank you all for help :) |
| 09:38 |
Dyrcona |
No reason at all for the second to have the state='error'. |
| 09:39 |
Bmagic |
jvwoolf++ # love that battle cry |
| 09:39 |
Dyrcona |
dicreckard: I recommend you try this: https://gist.github.com/Dyrcona/00bd6b6290b6fbbb579c7f93b360ab0d It will create a database upgrade script between two arbitrary Evergreen versions. It requires git, and I guarantee the script will not work on the first try. |
| 09:56 |
Dyrcona |
Both events have the same update_time: 2024-03-21 05:43:13.394412-04. I think that might be significant. That they're being updated at the same time. |
| 10:04 |
Bmagic |
terranm++ # lol, good to know that yall know what I would type so you know when to be suspicious hahaha |
| 10:05 |
terranm |
Bmagic++ |
| 10:10 |
Bmagic |
csharp_: https://bugsquash3.mobiusconsortium.org is up and running at your service. I did login and test placing a hold and it worked! bugsquash2 has several patches that are candidates for having broke holds. I'll figure out which one and post on the bug |
| 10:15 |
|
kworstell_isl joined #evergreen |
| 10:16 |
|
kworstell_isl_ joined #evergreen |
| 10:24 |
Dyrcona |
ooh. got a real dead lock in a test system on Sunday: Process 3401997 ... blocked by process 3402007. ...Process 3402007 ... blocked by process 3401997. |
| 10:26 |
Dyrcona |
Looks like the hold targeter blocked a process trying to reindex the action.hold_request table and vice versa. |
| 10:26 |
* Dyrcona |
changes the hold targeter schedule. |
| 10:27 |
|
kworstell-isl joined #evergreen |
| 10:48 |
|
sleary joined #evergreen |
| 10:49 |
Rogan |
dickreckard I'm late to the party but echo jboyer's statement, just exporting isolated tables and reimporting them will be a bigger pain than the sql upgrades and you will lose a lot of data |
| 11:01 |
Dyrcona |
pg_dump/pg_restore usually does not work if the schema has changed. |
| 11:02 |
kenstir |
Can I get a clue how to set up my test server for Stripe payments? I have a stripe account, and I used the library settings editor to add things to BR1, but the BR1 patron with a fine does not see any "pay fines" button. Screenshot: https://launchpadlibrarian.net/720500896/2024-03-20-Library-Settings-Editor.png |
| 11:04 |
Dyrcona |
kenstir: You added your test settings for stripe and enabled stripe as the cc processor? If so, you should be able to "pay" with one of the stripe test cards. See the strip documentation. |
| 11:05 |
kenstir |
Dyrcona Yes I did but the issue is I don't see any pay button in EG when logged in as the patron. I do see the fine. |
| 11:06 |
Dyrcona |
you restarted services and apache2 after making the settings changes? they don't always have an immediate effect. |
| 11:06 |
mmorgan |
kenstir: Did you set the library setting Allow Credit Card Payments (credit.payments.allow) to TRUE also? |
| 14:10 |
berick |
Dyrcona: yeah, and I'll probably be poking at https://opensearch.org/ before too long as an Elastic replacement. |
| 14:12 |
jeffdavis |
Are we ok to officially recommend disabling JIT in Postgres? I don't recall if anyone had reservations about turning it off. (bug 2042158) |
| 14:12 |
pinesol |
Launchpad bug 2042158 in Evergreen "Postgres 12+ needs to have JIT disabled" [Undecided,Confirmed] https://launchpad.net/bugs/2042158 - Assigned to Galen Charlton (gmc) |
| 14:18 |
Dyrcona |
jeffdavis: I have reservations about turning it off, but I have not made them public. If we have queries that cause issues with the JIT we should probably fix the queries. I have not turned off JIT in my Pg 15 and Pg 16 test databases and I've not encountered the issue described in the bug. All of that said, don't let just me stop anything from going forward. |
| 14:23 |
Dyrcona |
Speaking of PostgreSQL settings, this is mainly for Bmagic: I think I'm going back to 1.1 for random_page_cost. A couple of queries that I've run since setting it to 1.0 seem slower. |
| 14:25 |
Dyrcona |
... on Pg10 anyway. |
| 14:40 |
JBoyer |
I haven't weighed in much on that or had much time to do much lately but feel the same as Dyrcona re: turning off the JIT unconditionally. |
| 09:29 |
* mmorgan |
tries to grab any of kmlussier's tuit cookies that remain |
| 09:29 |
Dyrcona |
Huh.... websocketd-osrf.service got disabled somehow. I've seen systemd disable user services after an update, but not system services.... |
| 09:30 |
Dyrcona |
systemd-- |
| 09:35 |
Dyrcona |
Nooooo! All that work and the test system still looks different from production. I even remade a new test branch in exactly the same way as production. |
| 09:35 |
Dyrcona |
i wonder if there's a template change in production that doesn't appear in the override folders? That has to be it. |
| 09:39 |
Dyrcona |
Nope. I just copied the production templates to my custom branch and `git diff` does nothing. |
| 09:47 |
Dyrcona |
git status say everything up to date. |
| 10:10 |
|
jvwoolf joined #evergreen |
| 10:10 |
|
mmorgan joined #evergreen |
| 10:11 |
Dyrcona |
This sort of thing drives me nuts. |
| 10:29 |
Dyrcona |
OK, so it looks like the difference is that templates/bootstrap/opac/parts/cart_nav.tt2 is displaying on the test system but not in production, even though it looks like it ought to display on both given what the templates all look like..... |
| 10:32 |
|
Christineb joined #evergreen |
| 10:36 |
csharp_ |
templates-bootstrap, you mean? |
| 10:36 |
csharp_ |
(wondering if the path is wrong) |
| 10:37 |
Dyrcona |
csharp_: You're correct. I can't type in chat. |
| 10:38 |
csharp_ |
btw, I'll state for the record that I don't like having templates and templates-bootstrap - seems like the bootstrap versions could be an option to symlink to or something and just have "templates", but it's ok |
| 10:38 |
Dyrcona |
What it looks like is production is rendering templates-bootstrap/opac/parts/topnav_subnav.tt2 differently than the test machine. |
| 10:38 |
csharp_ |
gotcha |
| 10:39 |
csharp_ |
@who is in charge of writing things down in The Record? |
| 10:39 |
pinesol |
eeevil is in charge of writing things down in The Record. |
| 10:42 |
JBoyer |
Dyrcona, any chance apache config stuff is an issue? |
| 10:43 |
Dyrcona |
JBoyer: The overrides are in different paths, but the Apache stuff is pretty much the same. |
| 10:44 |
Dyrcona |
I've eyeballed the eg_vhost.confs, but I'll diff them. |
| 10:47 |
Dyrcona |
OK. i difference: templates_cons (the override) comes before templates-bootstrap in production. On my test system both of the overrides come after the stock directories. I'll swap those and see what happens. |
| 10:47 |
Dyrcona |
s/i/1/ |
| 10:48 |
Dyrcona |
JBoyer++ That was it! |
| 10:49 |
Dyrcona |
csharp_++ too. |
| 11:03 |
|
kenstir joined #evergreen |
| 11:04 |
terranm |
As long as we close more bugs than we open |
| 11:19 |
|
mantis joined #evergreen |
| 11:19 |
mantis |
getting a 500 internal error when signing into the OPAC on MOBIUS test server 1 |
| 11:24 |
pinesol |
News from commits: LP#2052567: allow Reports to load when BitWarden plugin is active <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=d3c203d3a1875f74a0ffe2db4431b49eb4564d3c> |
| 11:27 |
|
jvwoolf1 joined #evergreen |
| 11:34 |
Bmagic |
mantis: I might have been restarting apache at that moment (updating it for Czech language) - it's working for me at the moment |
| 11:36 |
mantis |
got it! thanks! |
| 11:37 |
csharp_ |
working on testing bug 1525394 and I have verified that part-holds now work, but we don't use serials and I'm not sure how to test issuance holds :-/ |
| 11:37 |
pinesol |
Launchpad bug 1525394 in Evergreen "SIP patron part level holds respond blank" [Undecided,Confirmed] https://launchpad.net/bugs/1525394 - Assigned to Chris Sharp (chrissharp123) |
| 11:38 |
csharp_ |
Bmagic: do you know if issuance holds are possible on your testbox with that fix on it? |
| 11:39 |
Bmagic |
I'm fuzzy on "issuance holds" but I don't see why the system wouldn't be able to do it. It's mostly stock Evergreen main |
| 11:47 |
Bmagic |
I spelled a couple of those wrong, lol |
| 11:49 |
Bmagic |
terranm: https://bugsquash2.mobiusconsortium.org/eg2/en-US/staff/catalog/search?org=1&limit=10&query=a&fieldClass=keyword&joinOp=&matchOp=contains&format=serial&dateOp=is&ridx=0 |
| 11:56 |
jeffdavis |
I'm a little disappointed to learn that Cat Fancy magazine has rebranded as Catster |
| 11:58 |
csharp_ |
Bmagic: actually I've been testing on our own server, but thanks for the connection info - I'll give yours a shot |
| 11:59 |
csharp_ |
jeffdavis++ |
| 12:00 |
terranm |
Bmagic++ jeffdavis++ |
| 12:01 |
collum |
Bmagic: in the url change query=a to query=* |
| 12:01 |
collum |
There's a few more. |
| 15:20 |
|
kmlussier joined #evergreen |
| 15:22 |
|
smayo joined #evergreen |
| 15:31 |
|
mantis left #evergreen |
| 15:45 |
csharp_ |
ok, next time anyone asks why the bugs aren't getting signed off on, please remember that it can take a lot of time and effort just to set up a test case! |
| 15:46 |
csharp_ |
I've spent 2+ hours today just trying to figure out enough serials to create an issuance hold to test the "no part holds showing up in SIP2" bug :-/ |
| 15:46 |
csharp_ |
(still not there) |
| 15:47 |
csharp_ |
and it might be sunk cost fallacy at this point, but I'll be damned if I don't figure this out and commit the fix |
| 15:48 |
|
kworstell-isl joined #evergreen |
| 15:50 |
Dyrcona |
csharp_++ |
| 15:51 |
Dyrcona |
I'm in a similar boat with the SFTP EDI stuff. I'm trying to figure out enough acq to set up test orders and pregenerated responses. |
| 15:56 |
csharp_ |
where is tlittle when we need her in this hour of need? |
| 15:57 |
csharp_ |
Bmagic: is all well with the bugsquash2 server? I'm suddenly stuck |
| 15:57 |
* csharp_ |
weeps |
| 15:59 |
Bmagic |
try hitting it from another VM to confirm? |
| 16:04 |
csharp_ |
Bmagic: looks ok now - thanks for checking |
| 16:04 |
Bmagic |
ok great |
| 16:17 |
csharp_ |
Bmagic: on bugsquash2, when I try to place a hold on Catster magazine after creating a holdable copy, I'm getting an empty red error box and this in the console: "ERROR TypeError: t.lastRequest.result.evt is null" - can you confirm? |
| 16:17 |
csharp_ |
I'm able to place the hold on my own test server running main-ish |
| 16:17 |
Bmagic |
checking |
| 16:17 |
csharp_ |
placing the hold for Sarah Smith |
| 16:18 |
csharp_ |
holdable copy barcode is "chrisistesting" |
| 16:19 |
csharp_ |
(my own server doesn't have the SIP2 port open on the ITS-administered firewall, so I can't test there) |
| 16:20 |
csharp_ |
however, I guess I'm now wondering if the item I attached is in fact an "issuance" |
| 16:20 |
Bmagic |
hold rule matrix is in the way I think |
| 16:20 |
csharp_ |
ah |
| 16:20 |
Bmagic |
result: Object { success: false, evt: null } |
| 16:46 |
Bmagic |
bugsquash3 will end up being a tomorrow thing if you'll still be around? |
| 16:47 |
csharp_ |
yeah, I'll be here - no biggie |
| 16:47 |
Bmagic |
I'm excited about sqaushing this one! |
| 16:47 |
csharp_ |
and I have a PINES-ish server to test - but it probably needs even more serials setup than enhanced concerto since we've never used it |
| 17:09 |
|
mmorgan left #evergreen |
| 17:15 |
|
sleary joined #evergreen |
| 18:59 |
|
jihpringle joined #evergreen |
| 15:01 |
Bmagic |
#info mmorgan will explore moving LP stats to community site and automating same |
| 15:01 |
mmorgan |
Please carry forward. Wanted to also note that some of today's stats came from the Launchpad API. |
| 15:02 |
Bmagic |
#action mmorgan will explore moving LP stats to community site and automating same |
| 15:02 |
Bmagic |
#info sandbergja will see if gh actions can run the pgtap tests |
| 15:02 |
sandbergja |
I have a pullrequest for that, bug 2055796 |
| 15:02 |
pinesol |
Launchpad bug 2055796 in Evergreen "Have github actions run pgtap tests for us" [Undecided,New] https://launchpad.net/bugs/2055796 |
| 15:02 |
Bmagic |
sandbergja++ |
| 15:02 |
smayo |
sandbergja++ |
| 15:02 |
dluch |
sandbergja++ |
| 15:05 |
|
collum joined #evergreen |
| 15:05 |
Bmagic |
#action gmcharlt - create a Git commit message type and update bug 2051946 |
| 15:05 |
pinesol |
Launchpad bug 2051946 in Evergreen "institute a Git commit message template" [Wishlist,New] https://launchpad.net/bugs/2051946 - Assigned to Galen Charlton (gmc) |
| 15:05 |
Dyrcona |
Looks like it is in progress. I tested the other branch that makes release notes entries. |
| 15:05 |
eeevil |
gmcharlt_ is traveling today, aiui. |
| 15:05 |
Bmagic |
ah, cool |
| 15:05 |
Bmagic |
#info Stompro will formalize the tense usage in the release-note message |
| 15:13 |
sandbergja |
Dyrcona++ |
| 15:13 |
eeevil |
re the first part, I think that's smart (and then the release can get the squashed bugs) |
| 15:13 |
Bmagic |
+1 (and I can help with the building(s)) |
| 15:13 |
abneiman |
+1 to 27th, I can test & eval the release-notes script as part of point releases |
| 15:13 |
mmorgan |
+1 to March 27, I can help also. |
| 15:14 |
sandbergja |
Bmagic++ |
| 15:14 |
sandbergja |
abneiman++ |
| 15:23 |
Bmagic |
#topic New Business - Barriers to getting things committed |
| 15:23 |
jeffdavis |
I can start this off |
| 15:23 |
Bmagic |
please do |
| 15:23 |
jeffdavis |
I want to commit more pullrequests, but when I try, I often run into the same barriers: |
| 15:23 |
jeffdavis |
(1) no test environment available, (2) no test plan, (3) test plan is difficult to set up, (4) merge conflicts, esp with code that has sat uncommitted for months, (5) extra overhead required to backport and/or unsure whether to backport, (6) unresolved questions about the fix. |
| 15:23 |
jeffdavis |
I wonder if there are things we can be doing to mitigate some of those barriers? |
| 15:23 |
jeffdavis |
For example, would more community dev VMs be helpful? |
| 15:24 |
Dyrcona |
For 5 we can backport fewer fixes, particularly those that touch the database. |
| 15:24 |
Bmagic |
I think the answer is: yes |
| 15:24 |
Bmagic |
do we need a system for people to "checkout" a VM so it's their's for a time? |
| 15:25 |
sandbergja |
It seems that if we address some of the others, 4 might take care of itself (i.e. if we have a quicker commit cadence, branches won't sit for as long) |
| 15:25 |
abneiman |
4 is a problem, but especially a chicken-and-egg thing since the longer things sit without review the more conflicts they accumulate. For 2, I can commit to sharing test plans for Equinox-developed features. |
| 15:25 |
Bmagic |
we could use Evergreen to manage the checkouts |
| 15:25 |
abneiman |
sandbergja: great minds, lol |
| 15:25 |
Dyrcona |
i also think it is perfectly fair to comment on the bug that there is no test plan provided, and it's not obvious how to test the bug. |
| 15:26 |
Dyrcona |
For 4, it's also perfectly fine to ask the original developer to rebase it, or at least comment that it needs a rebase if you're not comfortable doing it yourself. |
| 15:26 |
sleary |
If you are the person rebasing it, and the merge conflicts have to do with CSS or ARIA, please ping me here and I'll be happy to help. |
| 15:27 |
abneiman |
+1, asking for dev rebases is totally fair game |
| 15:27 |
abneiman |
sleary++ |
| 15:27 |
jeffdavis |
I've also shared pullrequests and had to rebase them multiple times, which gets frustrating, so asking the dev to rebase is fair but only goes so far IMO. |
| 15:28 |
abneiman |
so it sounds like we really have 4 communications problems, 1 technical problem (test environments), and 1 practice problem (when do we backport?) |
| 15:28 |
Dyrcona |
jeffdavis: I do try to handle the rebases myself, but sometimes, its not obvious how to resolve it. |
| 15:29 |
mmorgan |
abneiman++ |
| 15:29 |
jeffdavis |
abneiman: great point |
| 15:29 |
kmlussier |
Often, when somebody asks for a rebase, it's during a rare moment when the tester has time to look at the code. If the person doesn't rebase it quickly, that person may no longer be available to test. Not so much a communications problem, but a tuits problem. |
| 15:30 |
Dyrcona |
I think that's more of a time problem. Most of us have "other jobs" or at least our job has more requirements than working on Evergreen code. |
| 15:30 |
dluch |
abneiman++ |
| 15:31 |
abneiman |
perhaps if a tester and a dev had a quick conversation, though, everyone's time could be used more valuably - "hey I'm planning to test this in $timeframe, do you mind looking at a rebase?" "I can do rebase within $possibleothertimeframe and let you know when it's done" etc etc |
| 15:32 |
mmorgan |
I think the code review sessions have been great for getting folks together to tackle individual bugs, whatever their issues. |
| 15:32 |
abneiman |
code review is GREAT for this |
| 15:32 |
abneiman |
sandbergja++ |
| 15:39 |
Bmagic |
:), nevermind on the winding down |
| 15:39 |
kmlussier |
Oh, my. That's an old bug. I even have comments on it. |
| 15:39 |
abneiman |
Dyrcona: curious if you have an alternate suggestion? |
| 15:40 |
mmorgan |
Regarding test environments - community ones that can be checked out exclusively would be great |
| 15:40 |
terranm |
Maybe we need to have a needsdiscussion cleanup party, too |
| 15:40 |
redavis |
Bmagic++ mmorgan++ |
| 15:40 |
sleary |
terranm++ |
| 15:45 |
dluch |
We might be able to help with VMs...have to discuss internally, though |
| 15:45 |
abneiman |
Monthly half-day open code reviews or the like, with rotating responsibilty for hosting / VMs? |
| 15:45 |
csharp_ |
Bmagic: for me/us, the networking piece is an issue - PINES/GPLS machines are behind a finicky firewall :-/ |
| 15:45 |
Dyrcona |
We're supposed to be requiring test plans and release notes, so enforce that. (I've been adding them for my recent branches.) |
| 15:45 |
csharp_ |
but we can talk logistics at that level later |
| 15:45 |
abneiman |
and a committment from each senior committer to attend 1 per year or something |
| 15:46 |
kmlussier |
abneiman: I think a middle way is desireable. Smaller and more consistent contributions is a better approach than a mass of contributions / review happening at the same time. |
| 15:49 |
csharp_ |
it's not just a matter of time/tuits - I just need to develop better habits |
| 15:49 |
abneiman |
I'm just trying to think about ways to spread the load - if more people are doing things, we're relying less on the community unicorns (you know who you are lol) to shoulder so much |
| 15:50 |
dluch |
++ |
| 15:50 |
Dyrcona |
I've also been burned by not testing some big things thoroughly enough, so I like to set aside at least a day to test even small things. |
| 15:50 |
redavis |
(if you're in this meeting, you're a community unicorn) |
| 15:50 |
Bmagic |
redavis++ |
| 15:50 |
sleary |
kmlussier gmcharlt_ went over things with me, but I don't think there is much written down in the wiki on going from contributor to committer |
| 15:51 |
csharp_ |
🦄🦄🦄 |
| 15:51 |
kmlussier |
redavis: No, I'm just here because I like talking to all of you. |
| 15:51 |
Bmagic |
we're coming up on our hour yall |
| 15:51 |
sandbergja |
Dyrcona made me think of something that would be helpful for me: if there is a way we could run the tests against each pull request automatically. |
| 15:51 |
Dyrcona |
kmlussier: If you do want the commit bit back, just let me know. I can do it without a vote. :) |
| 15:51 |
sandbergja |
That green checkbox in Github saying "your tests passed" really helps me in other open source projects |
| 15:52 |
kmlussier |
redavis brought up earlier the idea of talkng about this at the hackfest. I've been thinking it might be worthwhile to have a monthly meeting where we could focus on one problem we want to solve. Because we could talk about this all day. |
| 15:52 |
Bmagic |
sandbergja: yes! a container that lauches with a branch and runs the test and dumps the results |
| 15:52 |
* csharp_ |
feels us teetering on the edge of the "move Git" discussion - keep his mouth shut |
