--quote-all on empty file with only headers yield output without headers

[johanmazelanssi]

Attached files

• not_ok.csv
• ok.csv

Expected behavior

When I use the "--quote-all" flag and the "cat" verb, I expect to get all cells with double quote around the content.

Example:
mlr --csv --quote-all cat "ok.csv"

Observed behavior

When a file contains csv headers but no data, the output is empty.

Example:
mlr --csv --quote-all cat "not_ok.csv"

Question

Is this an expected behavior?
Did I miss a flag that fix that would allow me to get my expected output?

[aborruso]

Hi,
for your not ok and in some way wrong csv file, you could run

mlr --csv -N --quote-all cat "not_ok.csv"

to get

"name","age"

[johanmazelanssi]

Thank you very much that indeed fixes my problem. :)

Shouldn't the default behavior not require this flag though?

[aborruso]

@johnkerl can answer your question here.

Miller is a tool for transforming structured text. A header row without data is perhaps not the type of data Miller was designed for.

[johnkerl]

Hi @johanmazelanssi and @aborruso !

This is an instance of #314.

Miller doesn't handle a stream containing zero records.

• DKVP lines have key-value pairs, like a=1,b=2,c=3 on one line, then a=4,b=5,c=6 on another.
• CSV (with header), by contrast, has keys all on the first line, like a,b,c, then values on subsequent lines like 1,2,3 and 4,5,6. When there is only the header line and no data lines, as far as Miller is concerned, there aren't any records at all.
• CSV without header is implicitly keyed. For example if your file has just the line a,b,c and you do mlr --icsv -N --ojsonl cat my.csv then you get {"1": "a", "2": "b", "3": "c"}. That is, a,b,c is a data line, not a header line, and the keys have been set to 1,2,3.

@aborruso trick with -N is a trick, but it shouldn't be the default. When -N is used, the first line is no longer the header line.

#314 is a good project to do, and a non-trivial one. Miller would need to have some concept of a zero-length record propagated throughout its source code.

[johanmazelanssi]

Thank you very much for the answer and the pointer.
I am sorry for not finding this issue before posting this thread.

Thank you for this tool. :)

[johnkerl]

No worries @johanmazelanssi !! #314 is very old (and still unresolved) -- not easy to find. :(