split file / separar archivo
Quote from joferar333 on March 17, 2023, 1:07 pm@vadim,@gaev
Hello friend, surely you can give me a hand since yous are really the neowin master here...
I have to build a small routine that takes a file that contains x lines... for example 20,000 and I need to separate the content of that file into files that contain x lines for example
the original file contains 20,000 lines
and the result would be 4 files of 5000 lines each....Español
Tengo que tomar un archivo que tiene por ejemplo 20000 lineas y deseo obtener de ese archivo 4 archivos que contentan 5000 lineas cada una..
Determinar la cantidad de lineas, armar el lopp barbaro pero me pierdo a la hora de la separacion y la escritura de los archivos...
Hello friend, surely you can give me a hand since yous are really the neowin master here...
I have to build a small routine that takes a file that contains x lines... for example 20,000 and I need to separate the content of that file into files that contain x lines for example
the original file contains 20,000 lines
and the result would be 4 files of 5000 lines each....
Español
Tengo que tomar un archivo que tiene por ejemplo 20000 lineas y deseo obtener de ese archivo 4 archivos que contentan 5000 lineas cada una..
Determinar la cantidad de lineas, armar el lopp barbaro pero me pierdo a la hora de la separacion y la escritura de los archivos...
Quote from Gaev on March 17, 2023, 4:01 pm@joferar333the solution depends on a couple of factors ...
1) what is the size/length of each line (this will determine if all the lines can be read into memory at one time); if the lines are not all the same, can you provide the size of the file ?
2) is this a one-time requirement or something to be performed daily/weekly (this will determine if you can withstand a relatively longer time to process) ?
@joferar333the solution depends on a couple of factors ...
1) what is the size/length of each line (this will determine if all the lines can be read into memory at one time); if the lines are not all the same, can you provide the size of the file ?
2) is this a one-time requirement or something to be performed daily/weekly (this will determine if you can withstand a relatively longer time to process) ?
Quote from joferar333 on March 17, 2023, 5:51 pm@gaev
The file to be separated always has a
number of equal characters per line
They are a list of phone numbers.The total file to separate has 230,000 lines
each line is 13 characters longThis is a non-recurring process, I just need to process and separate the source file at once into as many files that contain 50,000 lines per file.
The file to be separated always has a
number of equal characters per line
They are a list of phone numbers.
The total file to separate has 230,000 lines
each line is 13 characters long
This is a non-recurring process, I just need to process and separate the source file at once into as many files that contain 50,000 lines per file.
Quote from Gaev on March 17, 2023, 6:07 pm@joferar333
Thank you for the relevant information.
Later today, I will post one solution that will be fast(er) ...
1) read the entire file into memory
2) parse each line as an array item
3) for each 50,000 items ...
a) copy the contents to a memory variable
b) write the memory variable to a sub-file
Thank you for the relevant information.
Later today, I will post one solution that will be fast(er) ...
1) read the entire file into memory
2) parse each line as an array item
3) for each 50,000 items ...
a) copy the contents to a memory variable
b) write the memory variable to a sub-file
Quote from joferar333 on March 17, 2023, 6:41 pm@gaev
Thanks...
Bad i not idea to
2) parse each line as an array item
3) for each 50,000 items ...
a) copy the contents to a memory variable
b) write the memory variable to a sub-filei never work to array...
Thanks...
Bad i not idea to
2) parse each line as an array item
3) for each 50,000 items ...
a) copy the contents to a memory variable
b) write the memory variable to a sub-file
i never work to array...
Quote from Gaev on March 18, 2023, 12:02 am@joferar333
This is NOT the final solution; it gives you an idea about how to extract and store one quarter of the lines in another file
SetVar "[myTelFile]" "[PubDir]myTelFile.txt" ... read contents of entire file into variable FileRead "[myTelFile]" "All" "[myTelFileLines]" ... split into array (lines separated by CR LF) StrParse "[myTelFileLines]" "[#13][#10]" "[arrLine]" "[arrLineCount]" ... array items are now in variables [arrLine1], [arrline2] etc. ... approximate count of quarter of the lines Math "[arrLineCount]/4" "0" "[maxLoop]" ... clear variable to store first part of file SetVar "[part1Lines]" "" ... append each line as you iterate through the array items Loop "1" "[maxLoop]" "[thisLoop]" SetVar "[part1Lines]" "[part1Lines][arrLine[thisLoop]][#13][#10]" EndLoop ... now write the contents of the [part1Lines] variable to file SetVar "[firstPartFile]" "[PubDir]firstPartFile.txt" FileWrite "[part1Lines]" "All" "[firstPartFile]"Note that this is untested code as I do not have a test file.
Try it out, preferably with a smaller test file; if you encounter problems, please provide a smaller telephone file, and I will test it on my machine.
This is NOT the final solution; it gives you an idea about how to extract and store one quarter of the lines in another file
SetVar "[myTelFile]" "[PubDir]myTelFile.txt" ... read contents of entire file into variable FileRead "[myTelFile]" "All" "[myTelFileLines]" ... split into array (lines separated by CR LF) StrParse "[myTelFileLines]" "[#13][#10]" "[arrLine]" "[arrLineCount]" ... array items are now in variables [arrLine1], [arrline2] etc. ... approximate count of quarter of the lines Math "[arrLineCount]/4" "0" "[maxLoop]" ... clear variable to store first part of file SetVar "[part1Lines]" "" ... append each line as you iterate through the array items Loop "1" "[maxLoop]" "[thisLoop]" SetVar "[part1Lines]" "[part1Lines][arrLine[thisLoop]][#13][#10]" EndLoop ... now write the contents of the [part1Lines] variable to file SetVar "[firstPartFile]" "[PubDir]firstPartFile.txt" FileWrite "[part1Lines]" "All" "[firstPartFile]"
Note that this is untested code as I do not have a test file.
Try it out, preferably with a smaller test file; if you encounter problems, please provide a smaller telephone file, and I will test it on my machine.
Quote from Gaev on March 18, 2023, 12:18 am@joferar333
There are times when VisualNEOWin is not the best tool for the job; programs specifically developed for a single purpose do a better (faster; no scripting) job.
One example you might look into is GSplit (https://www.gdgsoft.com/gsplit); I have not used it but it is free to download.
From Google Search results ...
The best alternative is FCorp File & Folder Tools, which is free. Other great apps like Free File Splitter are FFSJ, GSplit, SplitFile and Batch File Split & Join. Free File Splitter alternatives are mainly Merge File Tools but may also be Duplicate File Finders or File Search Utilities.
You might look into one or more of these.
There are times when VisualNEOWin is not the best tool for the job; programs specifically developed for a single purpose do a better (faster; no scripting) job.
One example you might look into is GSplit (https://www.gdgsoft.com/gsplit); I have not used it but it is free to download.
From Google Search results ...
The best alternative is FCorp File & Folder Tools, which is free. Other great apps like Free File Splitter are FFSJ, GSplit, SplitFile and Batch File Split & Join. Free File Splitter alternatives are mainly Merge File Tools but may also be Duplicate File Finders or File Search Utilities.
You might look into one or more of these.