How handle parrallel jobs with long beets process import #72
Description
Hi,
My beets import process is very long :
First as usual beets search for candidates (musicbrainz, bandcamp etc...) and if a candidate match then :
- fetch metadata
- fetch art
- chroma
- replaygain
- copy in my media directory
If releases are imported one by one num_parallel_jobs = 1 there is no problem but this block all following imports if Betanin need a manual input as long as i resolve it so it's not the best way to import automatically.
if i set num_parallel_jobs = 4 for example, and if multiple import task are running in parallel i have the following issue :
[betanin] starting cli program
/torrents/seed/Music/Neil Young & Crazy Horse - Barn [Reprise 093624878438] (2021) flac (10 items)
Tagging:
Neil Young & Crazy Horse - Barn
URL:
https://musicbrainz.org/release/67908d6d-8dc7-4eb2-98ca-a1b1b75c2e54
(Similarity: 99.7%) (tracks) (Digital Media, 2021, XW, Reprise Records)
* Song Of The Seasons -> Song of the Seasons
* Change Ain't Never Gonna Come -> Change Ain’t Never Gonna (title)
* Shape Of You -> Shape of You
* Tumblin' Thru The Years -> Tumblin’ Thru the Years
* Don't Forget Love -> Don’t Forget Love
Traceback (most recent call last):
File "/usr/bin/beet", line 33, in
sys.exit(load_entry_point('beets==1.5.1', 'console_scripts', 'beet')())
File "/usr/lib/python3.9/site-packages/beets/ui/__init__.py", line 1284, in main
_raw_main(args)
File "/usr/lib/python3.9/site-packages/beets/ui/__init__.py", line 1271, in _raw_main
subcommand.func(lib, suboptions, subargs)
File "/usr/lib/python3.9/site-packages/beets/ui/commands.py", line 973, in import_func
import_files(lib, paths, query)
File "/usr/lib/python3.9/site-packages/beets/ui/commands.py", line 943, in import_files
session.run()
File "/usr/lib/python3.9/site-packages/beets/importer.py", line 340, in run
pl.run_parallel(QUEUE_SIZE)
File "/usr/lib/python3.9/site-packages/beets/util/pipeline.py", line 446, in run_parallel
raise exc_info[1].with_traceback(exc_info[2])
File "/usr/lib/python3.9/site-packages/beets/util/pipeline.py", line 311, in run
out = self.coro.send(msg)
File "/usr/lib/python3.9/site-packages/beets/util/pipeline.py", line 170, in coro
task = func(*(args + (task,)))
File "/usr/lib/python3.9/site-packages/beets/importer.py", line 1444, in user_query
apply_choice(session, task)
File "/usr/lib/python3.9/site-packages/beets/importer.py", line 1515, in apply_choice
task.add(session.lib)
File "/usr/lib/python3.9/site-packages/beets/importer.py", line 773, in add
self.record_replaced(lib)
File "/usr/lib/python3.9/site-packages/beets/importer.py", line 788, in record_replaced
dup_items = list(lib.items(
File "/usr/lib/python3.9/site-packages/beets/library.py", line 1526, in items
return self._fetch(Item, query, sort or self.get_default_item_sort())
File "/usr/lib/python3.9/site-packages/beets/library.py", line 1500, in _fetch
return super()._fetch(
File "/usr/lib/python3.9/site-packages/beets/dbcore/db.py", line 1094, in _fetch
flex_rows = tx.query(flex_sql, subvals)
File "/usr/lib/python3.9/site-packages/beets/dbcore/db.py", line 858, in query
cursor = self.db._connection().execute(statement, subvals)
sqlite3.OperationalError: database is locked
[betanin] program finished with exit status `0`
The first import task locked the beets database and all others imports will failed with exit status 0
do you know how i can solve this issue ? it's not easy to explain this issue i hope it's understandable enough
--
Ps: thank you very much for betanin it's an awesome tool