Using split command for Large txt file

[William McDavid]

So I’m trying to split a file that has over 20k HL7 messages but it’s just creating a file that is the same size. The command I’m using is split -5000 filename. I’m told this should create files with 5k messages in each file. So I should have approximately 4 files but I’m only getting the one that is the size as the original file.

Topic

[Robert Kersemakers]

What OS are you on?

On RHEL you need to use parameter -l if you deviate from the default 1000 lines. So

split -l5000 <filename>

should give you the result you need. Use

man split

for the manual.

• This reply was modified 5 years, 5 months ago by Robert Kersemakers.

Zuyderland Medisch Centrum; Heerlen/Sittard; The Netherlands

[William McDavid]

Unix AIX 6.1

This gave me the same result file splitting to a new file of the same size.

• This reply was modified 5 years, 5 months ago by William McDavid.

[David Barr]

Try putting a space in the command: split -l 5000. If that doesn’t work, then it’s probably a limitation of the AIX version of this command. I’ve run into issues with AIX and HPUX utilities not dealing well with files that contain really long lines. You may have to write a script to split the file apart.

[Robert Kersemakers]

Just a thought: maybe the command doesn’t split on lines because it can find only one.

If the file contains CL lenght-encoded HL7 messages with only <CR> separating the segments and no <LF> or <CR><LF> to indicate the end of a line, then the command is working as it should.

Have a look at the file with hcihexdump or notepad++ to see if this is the case. Or try to split a normal text-file and see if that works.

 

Zuyderland Medisch Centrum; Heerlen/Sittard; The Netherlands

[Author]

Replies