Evergreen ILS Website

IRC log for #evergreen, 2016-04-01

| Channels | #evergreen index | Today | | Search | Google Search | Plain-Text | summary | Join Webchat

All times shown according to the server's local time.

Time Nick Message
00:59 Newziky joined #evergreen
03:53 berick joined #evergreen
07:28 rjackson_isl joined #evergreen
08:33 Dyrcona joined #evergreen
08:37 Dyrcona berick | kmlussier: Should I load the branch on lp 1564685, instead of working/user/berick/browse​r-patron-reg-3b-bugs-wip?
08:37 pinesol_green Launchpad bug 1564685 in Evergreen "Angular patron editor post-2.10 misc. bugs" [Undecided,New] https://launchpad.net/bugs/1564685
08:37 * Dyrcona assumes so, but you know what assuming can do.
08:37 kmlussier Dyrcona: Yes, please load that one instead.
08:38 kmlussier Dyrcona++
08:38 Dyrcona OK.
08:43 mmorgan joined #evergreen
08:53 Dyrcona kmlussier: There's a conflict with the working/user/dyrcona/lp15644​02-evergreen-isbn13-function and current master.
08:53 Dyrcona The conflict is easy to resolve and you have a choice.
08:53 * kmlussier gets her coin ready for flipping
08:53 Dyrcona There's a console.debug() statement that was commented out in master.
08:53 Dyrcona It is still active in the wip branch.
08:53 Dyrcona I can choose either option.
08:54 Dyrcona If it is left active, the JavaScript console will get a lot of messages about what is filling grids added to it.
08:54 Dyrcona Dunno if you want that while testing or not.
08:55 kmlussier Dyrcona: I think you can leave it commented out at this point.
08:55 Dyrcona OK.
09:00 Dyrcona The branch is ready. Time to build it, and upgrade the database.
09:03 Newziky joined #evergreen
09:09 Dyrcona csharp++ # For ambiguous reasons. ;)
09:11 * Dyrcona brushes up on adding a limit to a circ matrix matchpoint while waiting on upgrade scripts.
09:16 Dyrcona Ah...that's interesting.
09:16 Dyrcona They have a limit. Apparently, they just want it changed.
09:42 mllewellyn joined #evergreen
09:43 collum joined #evergreen
09:48 yboston joined #evergreen
09:54 Dyrcona git commit --patch is nice when you want to split recent changes into multiple commits, except when what you want to split is in the same patch hunk.
10:04 Dyrcona Ok, I take that back. The e command in git commit --patch takes care of that rather well.
10:15 csharp git++
10:28 Bmagic where is the setting that makes naco_normalize_keep_comma instead of standard naco_normalize?
10:37 kmlussier Bmagic: config.metabib_field_index_norm_map maps config.metabib_field definitions to the normalizer that should be used.
10:37 Bmagic thanks!
10:53 Bmagic kmlussier: ok, so, the config.index_normalizer ID#3 specifies "NACO Normalize -- retain first comma" - in your config.metabib_field_index_norm_map - do you have rows for field 31 and 32 ?
10:56 kmlussier Bmagic: nope
10:57 Bmagic hmmm, without a norm mapping for field 31 ("title";"browse") and 32 ("series";"browse") it seems like it is stripping the comma
10:58 Christineb joined #evergreen
11:20 csharp acq question: is there any direct link between fund debits and PO line items?  if the PO activation errors out at any point, it needs to be "reset" and I'm trying to figure out how to remove (or offset) fund debits that should no longer exist
11:21 csharp it doesn't appear that there is a link between the two from my initial investigation
11:22 berick csharp: the link is between acq.lineitem_detail's
11:26 Bmagic kmlussier: I'm not sure stripping the comma is my problem. When the indexes are sorted, it's not respecting the space character even when the comma is removed.
11:26 Bmagic kmlussier: for example this is the order that the db returns "wood alix" "woodall" "wood amanda"
11:26 kmlussier Bmagic: Yeah, I think I said the other day I didn't think it was related to the comma
11:27 kmlussier But I don't know why it's sorting differently for you than it is for me
11:27 Bmagic select sort_value from metabib.browse_entry where id in(select entry from metabib.browse_entry_def_map where source in(1360318,856019,1287379))order by sort_value
11:27 Bmagic the database is sorting them... er wrong*
11:28 Bmagic what causes postgres to ignore the space character during the "order by" clause ?
11:30 Dyrcona Bmagic: Are you sure there is a leading space?
11:31 Bmagic its the space between "wood" and "alix"
11:31 Dyrcona Oh, spaces in the middle....
11:31 csharp berick: ah - thanks - I'll look there
11:31 Dyrcona I've never noticed that behavior, but I've never looked.
11:32 Bmagic yep, that query returns that order  "wood alix" "woodall" "wood amanda"
11:32 Bmagic "woodall" should go after "wood amanda" don't you think?
11:32 Dyrcona If is it an asciibetical sort, yes, I would expect that.
11:33 Dyrcona oy.... it is not is it.
11:33 Bmagic ?
11:33 bmills joined #evergreen
11:34 Bmagic do I have the wrong character set on the DB or something?
11:34 Dyrcona Doubt it would make much difference.
11:35 Dyrcona LC_COLLATE is probably the main driver here, but I could be wrong.
11:36 Dyrcona Ours is LC_COLLATE = 'C'
11:36 Dyrcona I'm going to run your query out of curiosity. Granted my records with those ids won't be the same, so the results are likely meaningless.
11:38 Dyrcona As I thought, my records with those ids don't throw up any flags.
11:39 Dyrcona Closest that I get is 'churchill a life' followed by 'churchill winston' which does what is expected.
11:39 Dyrcona I'll see if I can find records with those same names in them.
11:40 kmlussier Dyrcona: I looked at the browse sorting for all the MassLNC sites a couple of days ago and didn't see the same behavior Bmagic had encountered.
11:40 Callender joined #evergreen
11:40 Bmagic bash# locale  LC_COLLATE="en_US.UTF-8"
11:40 Dyrcona Bmagic: No, the LC_COLLATE for the database.
11:41 Dyrcona You can check that in PgAdmin by clicking the database name and looking in the SQL Pane to the lower right.
11:41 Bmagic CREATE DATABASE evergreen  WITH OWNER = postgres       ENCODING = 'UTF8'       TABLESPACE = pg_default       LC_COLLATE = 'en_US.UTF-8'       LC_CTYPE = 'en_US.UTF-8'       CONNECTION LIMIT = -1;
11:41 bmills joined #evergreen
11:42 Dyrcona Bmagic: LC_COLLATE and LC_CTYPE should be 'C' at least that is my understanding.
11:43 Bmagic so, I have to drop the DB and create it again?
11:43 Dyrcona You might be able to change it on an existing database. I don't know.
11:43 Bmagic en_US.UTF-8 seems legit though
11:44 Dyrcona Well, it is legit, but I believe there is a good reason to use C with Evergreen.
11:44 Dyrcona I'm a bit fuzzy on the details.
11:44 Bmagic no kidding, wow, how the heck did this happen
11:45 Callender joined #evergreen
11:46 Dyrcona CREATE DATABASE eg_main  WITH OWNER = superuser       ENCODING = 'UTF8'       TABLESPACE = pg_default       LC_COLLATE = 'C'       LC_CTYPE = 'C'       CONNECTION LIMIT = -1;
11:46 Dyrcona That's from our production database.
11:47 Dyrcona I vaguely recall discussing it back in 2010/2011 when I asked about using en_US.UTF-8, myself, but I could be making that up.
11:48 Bmagic I see the code right here Open-ILS/src/sql/Pg/create_database_extensions.sql
11:48 Bmagic CREATE DATABASE :db_name TEMPLATE template0 ENCODING 'UNICODE' LC_COLLATE 'C' LC_CTYPE 'C';
11:48 Dyrcona Yeah. It is possible to create the database in other ways, though.
11:49 Bmagic sheesh, when we upgraded postgres, I pg_dumped the DB and restored it.
11:49 Bmagic I wonder if it was like that before the dump
11:49 Dyrcona Most likely. I've never had those settings change as a result of restoring a dump.
11:50 Dyrcona Except for the name and owner, my development database is the same as production, and I restored it this morning from last night's production dump.
11:51 Bmagic looking back at my dump.sql file - CREATE DATABASE evergreen WITH TEMPLATE = template0 OWNER = postgres;
11:51 Bmagic it didn't specify those switches during the creation
11:52 Dyrcona Ah, could your template0 has them.
11:52 * Dyrcona cannot English.
11:52 Dyrcona ;)
11:52 Bmagic template0.... that is a file somewhere?
11:55 Bmagic well, this has all been enlightening, I've got some work to do
11:55 Bmagic Dyrcona++ kmlussier++
11:56 Dyrcona I was just looking, and I don't recall where the templates live. Somewhere in the postgres admin database, I think.
11:57 tsbere template0 is normally the "nothing loaded by default" template. Template1 is the "includes more than the bare pieces" possibly including "global" customizations template.
11:57 tsbere Also, any *database* can also be treated as a template, as far as I know...
11:59 Dyrcona Right. I believe I copied production once by using it as a template just to see if that worked.
11:59 Dyrcona That said, I didn't think that template0 and template1 are databases.
11:59 Dyrcona I also thought they could found in pg_catalog somewhere, but i'm not seeing them.
12:00 Dyrcona Someone in #postgres is bound to know.
12:00 jeffdavis our template0 uses en_US.UTF-8 for collate and ctype but restored pg dumps of our Evergreen db have collate & ctype = 'C'
12:00 jeffdavis psql -l should show them
12:00 csharp I've experience transaction wraparound warnings for template0/1 before, so they *are* databases as far as I understand
12:01 csharp you can vacuum them
12:01 Dyrcona jeffdavis++
12:01 csharp (which I do to avoid the xact wraparound issue)
12:01 Dyrcona So, mine show en_US.UTF-8 also, but the database is restored with C.
12:02 Dyrcona Yeah, looks like they are databases and PgAdmin hides them from you.
12:03 brahmina joined #evergreen
12:03 Dyrcona Cool. I (re)learned something. :)
12:03 Dyrcona Anyway, gonna get some lunch.
12:07 jeffdavis pg_dump -i -Fd -Z 9 -f /path/to/dumpfile --serializable-deferrable evergreen
12:07 jeffdavis ^ that's basically the command we use to create a dump of our db - note that it does not output a .sql file
12:08 jeffdavis for restore, we create a new database using Evergreen's create_database_extensions.sql script, then use pg_restore to load the dumped data into it
12:09 Bmagic jeffdavis: yeah, I was going to basically do those steps in testing today
12:10 Bmagic Which switches do you use for eg_db_config.pl ?
12:11 jeffdavis perl Open-ILS/src/support-scripts/eg_db_config --update-config --service all --create-offline
12:11 jeffdavis plus  user, password, host, port, and database
12:12 Bmagic right
12:12 Bmagic ok, thanks, that is helpful
12:13 jeffdavis happy to help, good luck!
12:18 jihpringle joined #evergreen
12:28 jihpringle joined #evergreen
12:40 miker Bmagic: you can change the collation of specific columns, or even just of indexes built on columns. no need to blow away your whole db
12:41 miker http://www.postgresql.org/docs/9.​1/static/indexes-collations.html
13:28 DPearl joined #evergreen
13:32 ericar joined #evergreen
13:39 Bmagic miker: Sounds shakey. It would be more comforting knowing that the whole db is setup as 'C' and not just some* columns. As I am not 100% sure which columns are important and what affects what
14:09 abowling joined #evergreen
14:44 ericar joined #evergreen
14:45 ericar joined #evergreen
14:50 ericar joined #evergreen
14:52 ericar joined #evergreen
14:54 ericar joined #evergreen
15:17 kmlussier berick: We have one more 2.8 release before it becomes security only, right?
15:18 berick kmlussier: yes
15:19 kmlussier Thanks!
15:19 berick well, at least one more.  i forget what the policy is exactly.
15:19 rhamby have a good weekend all
15:19 * Dyrcona asked about that earlier this week, 'cause I wasn't sure, either.
15:19 Dyrcona rhamby: You, also!
15:21 kmlussier This email says one month after two subsequent stable releases - http://list.georgialibraries.org/pipermai​l/open-ils-dev/2011-November/007746.html
15:22 kmlussier Google doesn't show me any updates to the policy in that email.
15:23 * kmlussier should look at old meeting minutes to see if it was actually voted upon and approved.
15:23 Dyrcona Well, that sound to me like 2.8.8 next month would be the last.
15:27 berick thanks kmlussier
15:27 berick i guess we'll know when 2.10.2 is cut
15:32 miker Bmagic: sorry, meetings. Not sure what's shakey about it ... it's a standard feature of postgres, and for other sort-ish columns we do other thing -- like normalize values with padding and such.  using a locale-aware collation is important for sorting anything but 7bit ascii correctly, and, indeed, for data integrity more generally beyond sorting. but, that's just one DBA's opinion.  we could, of course, instead simply normalize the browse sort column
15:32 miker to account the known difference between ascii and unicode sorting in a locale-aware environment.
15:36 kmlussier OK, that information was outdated. The revised policy was posted to the list a month later - http://markmail.org/message/53umdzope7nobtqb. It looks like a release gets bug fix support for 12 months after its release, at which point it moves to security release.
15:36 * kmlussier is going to put this on a wiki page
15:36 kmlussier If it's not there already
15:39 kmlussier It looks like 2.8.0 was released on March 31, 2015, so, yes, it looks like we get one more bug-fix release.
15:39 kmlussier Right back where we started.
15:55 * Dyrcona sings "Back where we started....He we go 'round again...."
15:56 Dyrcona heh. typo-fingers strike again.
15:58 jlitrell joined #evergreen
16:20 wjr joined #evergreen
16:21 geoffsams joined #evergreen
16:25 kmlussier http://wiki.evergreen-ils.org/doku.​php?id=dev:release_process:schedule
16:28 Dyrcona kmlussier++
16:29 Bmagic miker: so, in this one paticular case, I could drop the index and create it again with the correct locale option?
16:33 pinesol_green [evergreen|Mike Rylander] LP#1373601: Consider relevant characters before using word-boundary checks - <http://git.evergreen-ils.org/?p=​Evergreen.git;a=commit;h=e7c8785>
17:16 mmorgan left #evergreen
17:36 miker Bmagic: I think you'd want to set the collation on the column and then reindex it ... that should be enough, I believe. a test server could tell.  that said, I do think normalizing the sort value differently would be a good step to avoid this altogether ... like, replace spaces with +s or some such (more thought than I've just given it would be warranted, of course)
17:37 maryj joined #evergreen
17:39 Bmagic miker: thanks again!
17:49 Bmagic I guess I have to drop the column and add it again. alter table metabib.browse_entry alter column sort_value SET NOT NULL COLLATE 'C'; does not work. errors on "COLLATE"
18:16 miker bmagic: not at a computer, but there's a spell if you change the data type. just change it to be the same.

| Channels | #evergreen index | Today | | Search | Google Search | Plain-Text | summary | Join Webchat