Components of the Library Data Clean Up Project
Scope: Bibliographic data (i.e. bib, holding, item) cleanup is an essential activity to ensure data integrity and operational efficiencies. This page lists all of components for the FY22 project. Links to project component action plans lead to the Box folder for the project. This page will be updated to reflect new or updated information as the timeline advances.
Contact: Jacquie Samples Natalie Sommerville
Units: Metadata and Discovery Strategy Department and Resource Description Department
Date created: 07/29/2021
Date finalized: 07/15/2022
Priority Definitions
Level | Priority | Priority description |
|---|---|---|
P1 | Major Blocker/no work around Probable deadlines prior to May 27, 2022 | Broken data that means FOLIO will not work correctly if it is not cleaned up. |
P2 | Huge Hurdle/work around painful
Probable deadlines prior to June 30, 2022 (adjusted since FOLIO timeline shifted) | Errors in data that will be difficult or impossible to remediate in FOLIO on migration or immediately after migration. |
P3 | Data Enhancement/work around not sustainable Probable deadline time-frame to be determined | Data change not possible in FOLIO on migration or immediately after migration, but we could limp along. Fine for now, clean up would add value. |
P4 | Nice to do | Probably can't do in FOLIO, but won't really mess any process up. |
P5 | none known | Just don't forget this type of problem exists |
Project Components and Priorities
Priority | Status | Title/Name | Cleanup Need Description/Notes | Impacts/Outcomes | Size (S/M/L) | Stakeholder Group, or | |
|---|---|---|---|---|---|---|---|
P1 | Completed as of May 27, 2022, all local work is completed. The only task left is to request that OCLC clears Staging. That decision needs to be made. | P1-A Full Scale OCLC Data Sync -- (previously called reclamation) | Remediate NDD holding and title data between Aleph and OCLC | Benefits holdings data integrity; enables transitions to RapidILL and migration to FOLIO | L | Ros Raeford, Dracine | |
P1 | Completed in conjunction with P1-A. Manual post-work pending | P1-E 040 by language clean-up project Remaining unresolved titles will be cataloged in RDD or RLTS | There are full-level bibs in system which were created for language of cataloging other eng (040$b) which should be reviewed and replaced (may require significant manual review/cataloging). Related work: Presence of 040 $b (not eng) needs to be updated in cataloging documentation and processes. | Benefits bibliographic data integrity and migration to FOLIO | L (10,179 per WMS; 18,567 per Aleph) | RDD | |
P1 | Postponed as of 6/30/2022 In-Process post-work pending |
| Delete Gov Doc records for print records without items. Other "extra" records exist, suppress as needed. (i.e.: malformed holding codes) | Benefits holdings data integrity; enables transitions to RapidILL and migration to FOLIO | L | Jacquie, Natalie, Jianying | |
P1 | Completed as of Aug 30, 2021 |
| Before Full Scale OCLC Data Sync, | Benefits holdings data integrity; enables transitions to RapidILL and migration to FOLIO | L | Jacquie, Leeda | |
P1 | Completed as of May 27, 2022 | P1 -D Aleph 935 - reunite related titles (935)
As of June 30, decision made to not migrate remaining 935 data | Lok Saba debates is an example Have data-set | Benefits holdings data integrity, shared print retention and migration to FOLIO | L | MMIT | |
P1 | Completed as of May 27, 2022 In-Process in conjunction with P1-A, post-work pending
On-the-fly cleanup portion of the component is complete. | P1- F Delete On the Fly and/or Circ-Created records with no "real" barcode | Can't delete records in FOLIO
| Benefits holdings data integrity; enables transitions to RapidILL and migration to FOLIO | S (1190) | MMIT | |
P1 | Completed as of Oct. 12, 2021 | P1- G Gov Docs Resizing Batch withdrawals, Schedules A and D completed as of 7 Oct. 2021. | review needed in RDD due to data inconsistencies Have data-set, needs manual refinement | Benefits holdings data integrity; enables transitions to RapidILL and migration to FOLIO |
| Ros Raeford | |
P1 | Completed, provisional* Data cleanup complete as of 24 Jan 2022
| P1-H Items with sublibrary CHEM or VESIC (AskTech #5167) Follow up on all BES holdings (including Herbaria), these may come back to the libraries, but decision was delayed in 2020. *Decision on remaining collection is needed before transfer and final data cleanup possible, but planning suggests this will be done by January 2023, so cleanup aspect is done for now. | Ghost holdings in bound-withs. Are these old reserves? How do we ID and remove them? wsl=VESIC (17 records) Leeda has data and notes to share | Benefits holdings data integrity; enables transitions to RapidILL and migration to FOLIO |
| Erin Nettifee | |
P1 | Completed as of Oct. 12, 2021 | P1-I Lost materials Several sets of holdings need to be set to Lost. This includes IPS=LM, holding codes for "Off-Site Stacks," etc. When the only holding we had is Lost, holdings need to be removed from OCLC | LM meant "missing from LC Reclass", but put in use by DUL Circ and Law for other reasons, now LM means the same across DUL and professional school libraries. DUL 'LM's need to be changed to "LO" when material is not currently circulating. | Benefits holdings data integrity; enables transitions to RapidILL and migration to FOLIO | M | Ros, Jacquie, Andrea Loigman | |
P1 | Completed as of Sept.1, 2021 | P1-J Clean up holdings with PK3 "locations and call numbers vary" | wcl=PK3* Suppress holdings records? Delete if no connected order? What happens with any "real" items? | Benefits holdings data integrity; enables transitions to RapidILL and migration to FOLIO | M (19,473) | Natalie | |
P1 | Completed as of July 15, 2022 | P1-K Microfiche described on print holding records (need to transfer to microfiche collection) | cross check item material type (microformat) and "general" stacks in DOC sublibrary. (~34,300 Suppressed on 10/12) consider maps with same inconsistency | Benefits bibliographic data integrity and migration to FOLIO | L | Holly Chang, CRA | |
P1 | Completed as of Feb. 22, 2022 – no bearing on Data Sync | P1-L HOL 863 summary statement in multi-part mono holdings: LDR Pos 06=v | move to 866 statements Make sure that this is a retired practice | Benefits holdings data integrity; enables transitions to RapidILL and migration to FOLIO | L | Jacquie | |
P1 | Completed as of May 2021. | P1-M Lilly Locked Stacks materials without "D" barcodes | Review each tab and ensure that IPS and other data are correct. | Corrects data errors and simplifies migration to FOLIO | S | Ros Raeford | |
P1 | In-Process as of Oct.1, 2021 | P1-N Update bibliographic records for Gale electronic resources
| Records provided by Gale; not available from 360. Original records loads described print, not electronic resources. New file has been supplied. We need to check to see if OCLC holdings are set. Attempt to match on 901 NCCO number. Approximately 300,000 records were suppressed as of Oct. 11, 2021 Make sure to think about RL records where Gale URLs may be added in error. | Benefits holdings data integrity; enables transitions to RapidILL and migration to FOLIO | L (approx.270,000) | CRA, RDD | |
P2 | Completed as of January 2022 | P2-A Statistical Analysis needed to enable logical Gov Docs resizing projects | Natalie will set up meeting with Jacquie & Jianying to tease out what strategy should be pursued | Benefits holdings data integrity; |
| Natalie | |
P2 | Completed as of July 8, 2022 | P2-B DDC to LC long-tail | see planning request from Ros | Benefits item data integrity and access to these materials to Duke community. | S | Ros Raeford | |
P2 | Completed as of March 11, 2022 | P2-C Crash records | delete after analysis | Removing extraneous records simplifies FOLIO migration. | M (5) | MMIT | |
P2 | Completed as of June 3, 2022 | P2-D Bib 999 tags | Some Aleph 999 tags will migrate to FOLIO 996, some need to be deleted for data integrity | Facilitates migration to FOLIO |
| MMIT | |
P2 | Completed as of June 4, 2021 | P2-E Aleph to FOLIO error logs | Many lines in the log files. Need to be broken out into workable projects Dennis has created PostGres DB to enable error log review and planning. | Corrects data errors and simplifies migration to FOLIO |
| MMIT, Data WG, LSIS | |
P2 | Completed as of February 28, 2022 | P2-F Update MFHD summary statements for multi-part monographs | These data are not valid according to the ANSI/NISO Z39.71standard and should be updated both to confirm holding accuracy and to comply with this standard. | Benefits holdings data integrity; enables transitions to RapidILL and migration to FOLIO |
|
| |
P2 | Completed as of June 3, 2022 Now considered normal workflow | P2-G Clean up unused/unnecessary locations | Reduce the number of locations to create in FOLIO to facilitate migration example: Remove Quarto locations: We no longer use/apply locations that include a Q because we haven't used Quarto since at least 2012. RL may still use Quarto accurately (20,550) | Benefits holdings data integrity, shared print retention and migration to FOLIO | L | Erin Nettifee, Natalie | |
P2 | In-Process Postponed Change in collection decision at DKU, means that this component is on long-term hiatus.
| P2-H DKU Gift Loads (AskTech #5398) | Wuhan University gifted DKU library 50,000 titles with CNMARC records; June asked for help in transforming them into MARC21 | Benefits DKU and MarcEdit library community; priority set based on timeline of request | L | June Ma | |
P2 | Completed As of 3/7/2022 all item records are deleted | P2-I Material Type ITNET (AskTech #5983) | It looks like we can delete all of these items as well as the linked holdings records. Which means that we "should" be able to retire that location as well. | Removing extraneous records simplifies FOLIO migration. | S | Erin Nettifee | |
P2 | Completed as of June 3, 2022 | P2-J CHART material type review (AskTech #5981) | Mapping CHART to Visual Materials during FOLIO Migration. Leeda and I worked on collapsing material types in 2020 and I want to make sure that consistent decisions are made. I have already seen that some of these materials are not posters or charts, but microfilm/microfiche. So, some analysis and cleanup is needed. | Corrects data errors and simplifies migration to FOLIO | M | Erin Nettifee | |
P2 | Completed as of July 28, 2021 | P2-K Move items in item status 23 (Math bound periodicals) to item status 06 (AskTech #6258) | There was never any reason to have a separate bound periodicals item status for Math materials. | Corrects data errors and simplifies migration to FOLIO | S | Erin Nettifee | |
P2 | Completed as of July 31, 2020 | P2-L Items with blank item statuses (AskTech #5108) | "This is from the material type work. I’ve attached SQL output for items with no item status field to be reviewed and corrected." | Corrects data errors and simplifies migration to FOLIO | S | Erin Nettifee | |
P2 |