Files & archives manager: MQL recipes

Recipes

Write a daily log, then rotate it into a gzip archive
Back up a project folder in five archive formats
Read an INI configuration and convert it for a legacy tool
Product picture to a BMP label for a thermal printer
Store a base64 attachment, check it, read it, archive it
Retention policy: purge the daily exports older than 30 days
Dispatch an inbox folder by file type through file connections
Stream a large log line by line and keep only the errors
Delivery package with a SHA-256 manifest, verified after unzip
Mainframe fixed-width file to CSV with implied decimals

executed on MentDB

Write a daily log, then rotate it into a gzip archive

An application writes its daily log with a buffered writer. At midnight the script counts the lines, compresses the log to .gz, removes the original, checks that the archive restores the same number of lines and lists the archives of the folder.

Pipeline

01 · writeBuffered writerfile writer_add_line→02 · measureLines and bytesfile count_lines→03 · rotateGzip the logcompress gz→04 · verifyRestore and countcompress ungz→05 · listArchives in the folderfile dir_list_regex

Script

admin

#1. The log of the day, written line by line: add the end of line yourself;
file mkdir "tmp/rx_logs";
file writer_open "rx_w" "tmp/rx_logs/app_2026-11-01.log" false TEXT "utf-8";
for (-> "[i]" 1) (<= [i] 400) (++ "[i]") {
	file writer_add_line "rx_w" (concat "2026-11-01 INFO order " (+ 10000 [i]) " processed in " (+ 20 (* 3 [i])) " ms" [_n_]);
};
file writer_add_line "rx_w" (concat "2026-11-01 ERROR order 10401 payment refused" [_n_]);
file writer_flush "rx_w";
-> "[open]" (concat "{\"exist\": " (file writer_exist "rx_w") ", \"writers\": " (string regexp_replace (file writer_show) "\s" "") "}");
file writer_close "rx_w";
#2. Lines and size of the log;
-> "[log]" "tmp/rx_logs/app_2026-11-01.log";
-> "[lines]" (file count_lines [log]);
-> "[bytes]" (file size [log]);
#3. Rotate: gzip, then remove the original;
compress gz [log] (concat [log] ".gz");
file delete [log];
#4. Restore into another file, count again and read its first line;
compress ungz (concat [log] ".gz") "tmp/rx_logs/restored.log";
file reader_open "rx_r" "tmp/rx_logs/restored.log" TEXT "utf-8";
-> "[first]" (file reader_get_line "rx_r");
file reader_close "rx_r";
file writer_closeall;
concat "{\"writer\": " [open] ", \"lines\": " [lines] ", \"bytes\": " [bytes] ", \"gz_bytes\": " (file size (concat [log] ".gz")) ", \"ratio_pct\": " (math round (/ (* 100 (file size (concat [log] ".gz"))) [bytes]) 1) ", \"restored_lines\": " (file count_lines "tmp/rx_logs/restored.log") ", \"first_line\": \"" [first] "\", \"original_exists\": " (file exist [log]) ", \"archives\": " (string regexp_replace (file dir_list_regex "tmp/rx_logs" ".*\.gz" true false) "\s" "") "}";

Result

mentdb

{"writer": {"exist": 1, "writers": ["rx_w"]}, "lines": 401, "bytes": 19293, "gz_bytes": 2107, "ratio_pct": 10.9, "restored_lines": 401, "first_line": "2026-11-01 INFO order 10001 processed in 23 ms", "original_exists": 0, "archives": ["app_2026-11-01.log.gz"]}

Go further

file writer_add_line writes the text as is: append [_n_] to get one line per call
Close every stream of the session in one call with file writer_closeall

executed on MentDB

Back up a project folder in five archive formats

Before an upgrade, an admin takes a snapshot of a project folder and compares the archive formats. The script copies the folder, archives it as zip, tar, tar.gz, tar.bz2 and jar, compresses one file alone in bz2, restores every archive and checks that the line count of the sources is unchanged.

Pipeline

01 · snapshotCopy the folderfile copy_dir→02 · archivezip, tar, tar.gz, tar.bz2, jarcompress tarGz→03 · singleOne file in bz2compress bz2→04 · restoreExtract everythingcompress untarGz→05 · verifyCount the source linesfile count_line dir

Script

admin

#1. A small project and its snapshot;
file mkdir "tmp/rx_proj";
file mkdir "tmp/rx_proj/src";
file create "tmp/rx_proj/src/main.mql" (concat "#Entry point;" [_n_] "-> \"[x]\" 1;" [_n_] "[x];") "UTF-8";
file create "tmp/rx_proj/src/util.mql" (concat "#Helpers;" [_n_] "concat \"a\" \"b\";") "UTF-8";
file create "tmp/rx_proj/README.txt" "Project RX, version 1.4" "UTF-8";
file copy_dir "tmp/rx_proj" "tmp/rx_snap";
#2. Five archive formats, and the readme alone in bz2;
compress zip "tmp/rx_snap" "tmp/rx_bk.zip";
compress tar "tmp/rx_snap" "tmp/rx_bk.tar";
compress tarGz "tmp/rx_snap" "tmp/rx_bk.tar.gz";
compress tarBz2 "tmp/rx_snap" "tmp/rx_bk.tar.bz2";
compress jar "tmp/rx_snap" "tmp/rx_bk.jar";
compress bz2 "tmp/rx_snap/README.txt" "tmp/rx_readme.bz2";
#3. Restore everything;
compress unzip "tmp/rx_bk.zip" "tmp/rx_r/zip";
compress untar "tmp/rx_bk.tar" "tmp/rx_r/tar";
compress untarGz "tmp/rx_bk.tar.gz" "tmp/rx_r/tgz";
compress untarBz2 "tmp/rx_bk.tar.bz2" "tmp/rx_r/tbz2";
compress unjar "tmp/rx_bk.jar" "tmp/rx_r/jar";
compress unbz2 "tmp/rx_readme.bz2" "tmp/rx_r/README.txt";
#4. Sizes and checks;
concat "{\"bytes\": {\"zip\": " (file size "tmp/rx_bk.zip") ", \"tar\": " (file size "tmp/rx_bk.tar") ", \"tar_gz\": " (file size "tmp/rx_bk.tar.gz") ", \"tar_bz2\": " (file size "tmp/rx_bk.tar.bz2") ", \"jar\": " (file size "tmp/rx_bk.jar") "}, \"source_lines\": " (file count_line dir "tmp/rx_proj" ".mql") ", \"restored_lines\": {\"zip\": " (file count_line dir "tmp/rx_r/zip" ".mql") ", \"tar\": " (file count_line dir "tmp/rx_r/tar" ".mql") ", \"tgz\": " (file count_line dir "tmp/rx_r/tgz" ".mql") ", \"tbz2\": " (file count_line dir "tmp/rx_r/tbz2" ".mql") ", \"jar\": " (file count_line dir "tmp/rx_r/jar" ".mql") "}, \"top_folder_in_zip\": " (string regexp_replace (file dir_list "tmp/rx_r/zip") "\s" "") ", \"is_dir\": " (file is_directory "tmp/rx_r/tgz/rx_snap/src") ", \"readme\": \"" (string trim (file load "tmp/rx_r/README.txt")) "\"}";

Result

mentdb

{"bytes": {"zip": 724, "tar": 5120, "tar_gz": 285, "tar_bz2": 304, "jar": 724}, "source_lines": 5, "restored_lines": {"zip": 5, "tar": 5, "tgz": 5, "tbz2": 5, "jar": 5}, "top_folder_in_zip": ["rx_snap"], "is_dir": 1, "readme": "Project RX, version 1.4"}

Go further

The archive keeps the name of the folder: compress unzip restores rx_snap/... inside the target
For very small files, tar.gz and tar.bz2 headers cost more than they save: compare with file size on real data

executed on MentDB

Read an INI configuration and convert it for a legacy tool

An integration reads its settings from an INI file. The script reads values from the file and from a string, keeps a backup, appends a new section with an accented label, then converts the file to ISO-8859-1 for an old tool and checks the absolute path it will be given.

Pipeline

01 · createINI filefile create→02 · readValues by sectionfile ini→03 · appendBackup and new sectionfile append→04 · stringINI from a stringfile ini_str→05 · encodingUTF-8 to Latin-1file copy_format

Script

admin

#1. The configuration file;
file create "tmp/rx_app.ini" (concat "[database]" [_n_] "host=10.0.0.5" [_n_] "port=5432" [_n_] "name=erp" [_n_] "[mail]" [_n_] "sender=noreply@example.com") "UTF-8";
-> "[host]" (file ini "tmp/rx_app.ini" "database" "host");
-> "[port]" (file ini "tmp/rx_app.ini" "database" "port");
#2. Backup, then a new section with an accented label;
file copy_file "tmp/rx_app.ini" "tmp/rx_app.ini.bak";
file append "tmp/rx_app.ini" (concat [_n_] "[export]" [_n_] "label=Données clients" [_n_] "dir=tmp/rx_export");
-> "[label]" (file ini_str (file load "tmp/rx_app.ini") "export" "label");
#3. The old tool reads Latin-1: one byte per accented letter;
file copy_format "tmp/rx_app.ini" "UTF-8" "tmp/rx_app_latin1.ini" "ISO-8859-1";
#4. Absolute path for the tool, built from the server folder;
-> "[abs]" (concat (file cur_canonical_dir) "/tmp/rx_app_latin1.ini");
concat "{\"jdbc\": \"jdbc:postgresql://" [host] ":" [port] "/" (file ini "tmp/rx_app.ini" "database" "name") "\", \"label\": \"" [label] "\", \"lines\": " (file count_lines "tmp/rx_app.ini") ", \"backup_bytes\": " (file size "tmp/rx_app.ini.bak") ", \"utf8_bytes\": " (file size "tmp/rx_app.ini") ", \"latin1_bytes\": " (file size "tmp/rx_app_latin1.ini") ", \"absolute_path_exists\": " (file exist [abs]) ", \"pwd_is_abs_dir\": " (equal (file pwd) (file cur_abs_dir)) ", \"created_today\": " (string starts_with (file meta_data "tmp/rx_app.ini" "creationTime") (string left (file last_modified "tmp/rx_app.ini") 10)) "}";

Result

mentdb

{"jdbc": "jdbc:postgresql://10.0.0.5:5432/erp", "label": "Données clients", "lines": 9, "backup_bytes": 77, "utf8_bytes": 127, "latin1_bytes": 126, "absolute_path_exists": 1, "pwd_is_abs_dir": 1, "created_today": 1}

Go further

file append with its encoding parameter writes to a wrong path in this version: call it with two parameters
Read the server configuration itself with file ini "conf/server.conf" <section> <field>
file copy_format also repairs files received in Windows-1252: read with that encoding, write in UTF-8

executed on MentDB

Product picture to a BMP label for a thermal printer

A warehouse prints parcel labels on a thermal printer that only accepts BMP. The script resizes the product picture, rotates it, builds the BMP with text lines above and below, checks the file signature with a binary reader, and copies it with a binary writer for the print spool.

Pipeline

01 · resize64 x 64 picturefile image_resize→02 · rotateQuarter turnfile image_rotate_90→03 · labelBMP with text linesfile png_to_bmp→04 · signatureFirst bytesfile reader_get_bytes→05 · spoolBinary copyfile writer_add_bytes

Script

admin

#1. Resize the picture, then rotate it a quarter turn;
-> "[small]" (string regexp_replace (file image_resize "images/info.png" 64 64) "\s" "");
file b64_write [small] "tmp/rx_small.png";
-> "[rot]" (string regexp_replace (file image_rotate_90 "tmp/rx_small.png") "\s" "");
#2. BMP label: two lines above the picture, one below;
file png_to_bmp "[\"PARCEL 10401\", \"Dock 3\"]" "[\"97400 Saint-Denis\"]" [rot] "tmp/rx_label.bmp";
#3. Check the signature: a bitmap starts with BM;
file reader_open "rx_r" "tmp/rx_label.bmp" BINARY null;
-> "[magic]" (string decode_b64 (string trim (file reader_get_bytes "rx_r" 2)));
-> "[open]" (concat "{\"exist\": " (file reader_exist "rx_r") ", \"readers\": " (string regexp_replace (file reader_show) "\s" "") "}");
file reader_close "rx_r";
file reader_closeall;
#4. Copy for the print spool with a binary writer;
file writer_open "rx_wb" "tmp/rx_spool_10401.bmp" false BINARY null;
file writer_add_bytes "rx_wb" (string regexp_replace (file b64_read "tmp/rx_label.bmp") "\s" "");
file writer_close "rx_wb";
concat "{\"small_png_bytes\": " (file size "tmp/rx_small.png") ", \"signature\": \"" [magic] "\", \"reader\": " [open] ", \"label_bytes\": " (file size "tmp/rx_label.bmp") ", \"spool_bytes\": " (file size "tmp/rx_spool_10401.bmp") "}";

Result

mentdb

{"small_png_bytes": 817, "signature": "BM", "reader": {"exist": 1, "readers": ["rx_r"]}, "label_bytes": 22134, "spool_bytes": 22134}

Go further

file b64_read returns base64 with line breaks: remove them with string regexp_replace ... "\s" "" before decoding
Close all readers of the session at once with file reader_closeall

executed on MentDB

Store a base64 attachment, check it, read it, archive it

A customer sends the invoices to pay as a CSV attachment encoded in base64. The script writes it into an inbox folder, checks the file (exists, not a folder, size, creation date), reads it line by line to accept or reject each invoice, archives an identical copy with the date of the day and removes the inbox.

Pipeline

01 · storeDecode to a filefile b64_write→02 · checkExists, size, metadatafile meta_data→03 · readLine by linefile reader_get_line→04 · archiveDated copyfile copy_file→05 · verifySame bytesfile b64_read

Script

admin

#1. A customer sends the invoices to pay as a CSV attachment, received in base64;
-> "[att]" "aW52b2ljZTtjdXN0b21lcjthbW91bnQKRi0yMDI2LTExODtMZWNsZXJjIFNhaW50LURlbmlzOzE4NDAuNTAKRi0yMDI2LTExOTtSdW4gTWFya2V0OzsKRi0yMDI2LTEyMDtQaGFybWFjaWUgZHUgUG9ydDs2MTIuMDAKRi0yMDI2LTEyMTtHYXJhZ2UgUGF5ZXQ7OTc1LjI1";
file mkdir "tmp/rx_inbox";
file mkdir "tmp/rx_inbox/archive";
file b64_write [att] "tmp/rx_inbox/invoices.csv";
#2. Checks on the stored file;
-> "[f]" "tmp/rx_inbox/invoices.csv";
-> "[checks]" (concat "\"exists\": " (file exist [f]) ", \"is_dir\": " (file is_directory [f]) ", \"inbox_is_dir\": " (file is_directory "tmp/rx_inbox") ", \"bytes\": " (file size [f]));
#3. Read it line by line, reject lines without an amount;
file reader_open "rx_r" [f] TEXT "utf-8";
-> "[header]" (file reader_get_line "rx_r");
-> "[ok]" 0; -> "[ko]" ""; -> "[sum]" 0;
while (is not null (-> "[line]" (file reader_get_line "rx_r"))) {
	-> "[amt]" (atom get [line] 3 ";");
	if (type is_decimal [amt] 8 2) {-> "[ok]" (+ [ok] 1); -> "[sum]" (+ [sum] [amt]);} {-> "[ko]" (concat [ko] (atom get [line] 1 ";") " ");};
};
file reader_close "rx_r";
#4. Archive a copy, check it is identical, b64_read wraps lines every 76 chars, then clean;
file copy_file [f] (concat "tmp/rx_inbox/archive/invoices_" (date curdate) ".csv");
-> "[same]" (equal (string regexp_replace (file b64_read (concat "tmp/rx_inbox/archive/invoices_" (date curdate) ".csv")) "\s" "") [att]);
-> "[archived]" (file dir_list "tmp/rx_inbox/archive");
-> "[stamp]" (string left (file last_modified [f]) 10);
-> "[created]" (file meta_data [f] "creationTime");
file delete "tmp/rx_inbox";
concat "{" [checks] ", \"header\": \"" [header] "\", \"accepted\": " [ok] ", \"total\": " (math round [sum] 2) ", \"rejected\": \"" (string trim [ko]) "\", \"archive_identical\": " [same] ", \"archived\": " (string regexp_replace [archived] "\s" "") ", \"created\": \"" [created] "\", \"received_on_today\": " (equal [stamp] (date curdate)) ", \"inbox_left\": " (file exist "tmp/rx_inbox") "}"

Result

mentdb

{"exists": 1, "is_dir": 0, "inbox_is_dir": 1, "bytes": 153, "header": "invoice;customer;amount", "accepted": 3, "total": 3427.75, "rejected": "F-2026-119", "archive_identical": 1, "archived": ["invoices_2026-11-01.csv"], "created": "2026-11-01T13:18:56Z", "received_on_today": 1, "inbox_left": 0}

Go further

file b64_read wraps the base64 every 76 characters: remove the whitespace before comparing it
For a large file keep the reader open and process the lines in batches of 1000 with a counter

executed on MentDB

Retention policy: purge the daily exports older than 30 days

A nightly job writes one export per day named export_YYYYMMDD.csv. Apply the retention policy: keep 30 days, but always keep the exports of the last day of a month, and report what was purged and the space freed.

Pipeline

01 · namesDate from the file namestring regexp_extract→02 · ageAge in daysdate nb_day→03 · ruleMonth-end exceptiondate last_day_of_month→04 · purgeDelete and measurefile delete

Script

admin

#1. Simulate the exports of the last 3 months, ages in days;
-> "[today]" (date sysdate);
file mkdir "tmp/rx_p4i_exports";
json load "rx_p4i_ages" "[0, 3, 12, 29, 31, 45, 61, 75, 92]";
json parse_array "rx_p4i_ages" "/" "[a]" {
	-> "[d]" (date datediff [today] "DAY" [a]);
	file create (concat "tmp/rx_p4i_exports/export_" (string replace [d] "-" "") ".csv") (concat "date;orders" [_n_] [d] ";" (* 10 (+ [a] 1))) "UTF-8";
};
#Also the last day of the previous month and the one before;
-> "[eom1]" (date last_day_of_month (date datediff (date first_day_of_month [today]) "DAY" 1));
-> "[eom2]" (date last_day_of_month (date datediff (date first_day_of_month [eom1]) "DAY" 1));
file create (concat "tmp/rx_p4i_exports/export_" (string replace [eom1] "-" "") ".csv") "date;orders" "UTF-8";
file create (concat "tmp/rx_p4i_exports/export_" (string replace [eom2] "-" "") ".csv") "date;orders" "UTF-8";
#2. Apply the policy;
-> "[size0]" (file size "tmp/rx_p4i_exports");
-> "[purged]" 0;
-> "[kept_eom]" 0;
json load "rx_p4i_list" (file dir_list_regex "tmp/rx_p4i_exports" "export_[0-9]{8}\.csv" true false);
json parse_array "rx_p4i_list" "/" "[f]" {
	-> "[d]" (date format (string regexp_extract [f] "([0-9]{8})" 1) "yyyyMMdd" "yyyy-MM-dd");
	-> "[age]" (date nb_day [d] [today]);
	if (> [age] 30) {
		#3. A month-end export is kept for the yearly closing;
		if (equal [d] (date last_day_of_month [d])) {++ "[kept_eom]";} {
			file delete (concat "tmp/rx_p4i_exports/" [f]);
			++ "[purged]";
		};
	};
};
#4. Report;
concat "Exports found: " (json count "rx_p4i_list" "/") ", purged: " [purged] ", month-ends kept beyond 30 days: " [kept_eom] ", left: " (string regexp_count (file dir_list "tmp/rx_p4i_exports") "export_") [_n_] "Space freed: " (- [size0] (file size "tmp/rx_p4i_exports")) " bytes";

Result

mentdb

Exports found: 11, purged: 5, month-ends kept beyond 30 days: 1, left: 6
Space freed: 130 bytes

Go further

Schedule it every night with job add and a cron.
Archive the month-end exports with compress tarGz before the yearly purge.
Read the retention in days from a parameter with parameter get value.

executed on MentDB

Dispatch an inbox folder by file type through file connections

Files land in an inbox. Declare the inbox, the archive and the reject folders as file connections, then route each file by its extension (orders, invoices, unknown), and count what each folder received, without a single hard-coded path in the routing.

Pipeline

01 · declareThree file connectionscm set file→02 · resolveFolder of each connectioncm get→03 · routeCopy by extension, then deletefile copy_file→04 · countFiles per folderfile dir_list

Script

admin

#1. The folders are declared once as connections;
cm set "tuto_p4i_inbox" {execute "file.local.config.get" "[dir]" "tmp/rx_p4i_inbox";};
cm set "tuto_p4i_orders" {execute "file.local.config.get" "[dir]" "tmp/rx_p4i_orders";};
cm set "tuto_p4i_rejects" {execute "file.local.config.get" "[dir]" "tmp/rx_p4i_rejects";};
#2. A helper gives the folder of a connection;
json load "rx_p4i_c" (cm get "tuto_p4i_inbox"); -> "[in]" (json select "rx_p4i_c" "/localDir");
json load "rx_p4i_c" (cm get "tuto_p4i_orders"); -> "[ok]" (json select "rx_p4i_c" "/localDir");
json load "rx_p4i_c" (cm get "tuto_p4i_rejects"); -> "[ko]" (json select "rx_p4i_c" "/localDir");
file mkdir [in]; file mkdir [ok]; file mkdir [ko];
file mkdir (concat [ok] "/csv"); file mkdir (concat [ok] "/xml");
#The inbox receives a mix of files;
file create (concat [in] "/PO_1001.csv") "id;qty" "UTF-8";
file create (concat [in] "/PO_1002.CSV") "id;qty" "UTF-8";
file create (concat [in] "/ASN_77.xml") "<asn/>" "UTF-8";
file create (concat [in] "/readme.docx") "?" "UTF-8";
file create (concat [in] "/PO_1003.csv.part") "id;" "UTF-8";
#3. Route by extension, ignore the files still being written;
-> "[log]" "";
json load "rx_p4i_in" (file dir_list_regex [in] ".*" true false);
json parse_array "rx_p4i_in" "/" "[f]" {
	-> "[ext]" (string lower (string regexp_extract [f] "\.([A-Za-z]+)$" 1));
	if (equal [ext] "part") {-> "[dest]" "";} {
		if (or (equal [ext] "csv") (equal [ext] "xml")) {-> "[dest]" (concat [ok] "/" [ext]);} {-> "[dest]" [ko];};
	};
	if (not (is null or empty [dest])) {
		file copy_file (concat [in] "/" [f]) (concat [dest] "/" [f]);
		file delete (concat [in] "/" [f]);
		-> "[log]" (concat [log] [f] " -> " (string replace [dest] "tmp/rx_p4i_" "") [_n_]);
	};
};
#4. Count per folder;
json load "rx_p4i_n1" (file dir_list (concat [ok] "/csv"));
json load "rx_p4i_n2" (file dir_list (concat [ok] "/xml"));
json load "rx_p4i_n3" (file dir_list [ko]);
-> "[res]" (concat [log] "csv: " (json count "rx_p4i_n1" "/") ", xml: " (json count "rx_p4i_n2" "/") ", rejects: " (json count "rx_p4i_n3" "/") ", still in inbox: " (string regexp_replace (file dir_list [in]) "\s+" "") ", orders is a folder: " (file is_directory [ok]));
file delete [in]; file delete [ok]; file delete [ko];
cm remove "tuto_p4i_inbox"; cm remove "tuto_p4i_orders"; cm remove "tuto_p4i_rejects";
json unload "rx_p4i_c"; json unload "rx_p4i_in"; json unload "rx_p4i_n1"; json unload "rx_p4i_n2"; json unload "rx_p4i_n3";
[res];

Result

mentdb

PO_1002.CSV -> orders/csv
PO_1001.csv -> orders/csv
ASN_77.xml -> orders/xml
readme.docx -> rejects
csv: 2, xml: 1, rejects: 1, still in inbox: ["PO_1003.csv.part"], orders is a folder: 1

Go further

Start it on each new file with file_watcher start on the inbox folder.
Point the connections to another disk in production with cm set file: the routing code does not change.
Move the rejects to a partner over SFTP with sftp put.

executed on MentDB

Stream a large log line by line and keep only the errors

An application log is too large to be loaded in memory. Read it line by line with a reader, write only the ERROR and FATAL lines to a new file with a writer, flush regularly and count the lines by level.

Pipeline

01 · openReader and writerfile reader_open→02 · loopLine by linefile reader_get_line→03 · writeErrors only, flush every 2 linesfile writer_flush→04 · closeClose and countfile count_lines

Script

admin

#1. A log of 9 lines, written with a writer: add the line break yourself;
file writer_open "rx_p4i_w0" "tmp/rx_p4i_app.log" false TEXT "utf-8";
file writer_add_line "rx_p4i_w0" (concat "2026-10-31 08:00:01 INFO  api started on port 8443" [_n_]);
file writer_add_line "rx_p4i_w0" (concat "2026-10-31 08:02:13 WARN  slow query 2.4 s on orders" [_n_]);
file writer_add_line "rx_p4i_w0" (concat "2026-10-31 08:05:40 ERROR payment timeout order=7712" [_n_]);
file writer_add_line "rx_p4i_w0" (concat "2026-10-31 08:06:02 INFO  retry order=7712" [_n_]);
file writer_add_line "rx_p4i_w0" (concat "2026-10-31 08:06:03 ERROR payment refused order=7712 code=51" [_n_]);
file writer_add_line "rx_p4i_w0" (concat "2026-10-31 09:12:44 INFO  batch 311 imported" [_n_]);
file writer_add_line "rx_p4i_w0" (concat "2026-10-31 10:30:00 FATAL disk full on /data" [_n_]);
file writer_add_line "rx_p4i_w0" (concat "2026-10-31 10:30:05 INFO  shutdown" [_n_]);
file writer_add_line "rx_p4i_w0" (concat "2026-10-31 10:31:00 WARN  restart requested" [_n_]);
file writer_close "rx_p4i_w0";
#2. Reader on the log, writer on the error file;
file reader_open "rx_p4i_r" "tmp/rx_p4i_app.log" TEXT "utf-8";
file writer_open "rx_p4i_w" "tmp/rx_p4i_errors.log" false TEXT "utf-8";
-> "[open]" (concat "reader=" (file reader_exist "rx_p4i_r") " writer=" (file writer_exist "rx_p4i_w"));
json load "rx_p4i_lv" "{\"INFO\": 0, \"WARN\": 0, \"ERROR\": 0, \"FATAL\": 0}";
-> "[kept]" 0;
-> "[line]" (file reader_get_line "rx_p4i_r");
while (not (is null [line])) {
	#3. Level is the 3rd token;
	-> "[lvl]" (string regexp_extract [line] "^\S+ \S+ ([A-Z]+)" 1);
	json uobject "rx_p4i_lv" / [lvl] (+ (json select "rx_p4i_lv" (concat "/" [lvl])) 1) INT;
	if (or (equal [lvl] "ERROR") (equal [lvl] "FATAL")) {
		file writer_add_line "rx_p4i_w" (concat [line] [_n_]);
		++ "[kept]";
		if (== (math mod [kept] 2) 0) {file writer_flush "rx_p4i_w";};
	};
	-> "[line]" (file reader_get_line "rx_p4i_r");
};
#4. Close and check;
file reader_close "rx_p4i_r";
file writer_close "rx_p4i_w";
concat "While running: " [open] [_n_] "Lines per level: " (string regexp_replace (json doc "rx_p4i_lv") "\s+" "") [_n_] "Error file: " (file count_lines "tmp/rx_p4i_errors.log") " lines, first: " (string regexp_extract (file load "tmp/rx_p4i_errors.log" "utf-8") "^([^\n]*)" 1) [_n_] "Still open after close: reader=" (file reader_exist "rx_p4i_r") " writer=" (file writer_exist "rx_p4i_w");

Result

mentdb

While running: reader=1 writer=1
Lines per level: {"ERROR":2,"INFO":4,"FATAL":1,"WARN":2}
Error file: 3 lines, first: 2026-10-31 08:05:40 ERROR payment timeout order=7712
Still open after close: reader=0 writer=0

Go further

Read a binary file by blocks with file reader_open in BINARY mode and file reader_get_bytes.
Send the error file to the support with mail send when FATAL is not 0.
List the open readers and writers of the server with file reader_show and file writer_show.

executed on MentDB

Delivery package with a SHA-256 manifest, verified after unzip

Before sending a data delivery to a partner, write a manifest with the size and the SHA-256 of every file, zip the folder, then play the partner: unzip it elsewhere and verify every checksum against the manifest.

Pipeline

01 · filesFiles of the deliveryfile create→02 · manifestSize and SHA-256 per filestring sha_256→03 · zipZip the foldercompress zip→04 · verifyUnzip and comparecompress unzip

Script

admin

#1. The delivery;
file mkdir "tmp/rx_p4i_dlv";
file create "tmp/rx_p4i_dlv/customers.csv" (concat "id;name" [_n_] "C-12;Garage Hoarau" [_n_] "C-07;Pharmacie du Port") "UTF-8";
file create "tmp/rx_p4i_dlv/orders.csv" (concat "id;customer;amount" [_n_] "7001;C-12;420.00" [_n_] "7002;C-07;89.90") "UTF-8";
file create "tmp/rx_p4i_dlv/readme.txt" "Delivery 2026-W44, separator ;, UTF-8" "UTF-8";
#2. The manifest: name;size;sha256;
-> "[man]" "file;bytes;sha256";
json load "rx_p4i_fl" (file dir_list_regex "tmp/rx_p4i_dlv" ".*\.(csv|txt)" true false);
json parse_array "rx_p4i_fl" "/" "[f]" {
	-> "[p]" (concat "tmp/rx_p4i_dlv/" [f]);
	-> "[man]" (concat [man] [_n_] [f] ";" (file size [p]) ";" (string sha_256 (file load [p] "UTF-8")));
};
file create "tmp/rx_p4i_dlv/MANIFEST.csv" [man] "UTF-8";
#3. Zip the folder;
compress zip "tmp/rx_p4i_dlv" "tmp/rx_p4i_dlv.zip";
#4. The partner side: unzip and verify;
file mkdir "tmp/rx_p4i_partner";
compress unzip "tmp/rx_p4i_dlv.zip" "tmp/rx_p4i_partner";
-> "[root]" "tmp/rx_p4i_partner/rx_p4i_dlv";
-> "[ok]" 0;
-> "[ko]" 0;
csv parse (mql "M") (mql "tmp/rx_p4i_partner/rx_p4i_dlv/MANIFEST.csv") (mql ";") (mql "\"") (mql "") {
	if (equal (string sha_256 (file load (concat [root] "/" [M_file]) "UTF-8")) [M_sha256]) {++ "[ok]";} {++ "[ko]";};
};
concat "Zip: " (file size "tmp/rx_p4i_dlv.zip") " bytes, created " (string substring (file meta_data "tmp/rx_p4i_dlv.zip" "creationTime") 0 10) [_n_] "Manifest lines: " (- (file count_lines "tmp/rx_p4i_dlv/MANIFEST.csv") 1) [_n_] "Verified: " [ok] " ok, " [ko] " corrupted" [_n_] "orders.csv sha256: " (string substring (string sha_256 (file load (concat [root] "/orders.csv") "UTF-8")) 0 16) "...";

Result

mentdb

Zip: 998 bytes, created 2026-11-01
Manifest lines: 3
Verified: 3 ok, 0 corrupted
orders.csv sha256: ad093b48f9d7d567...

Go further

For binary files, hash the base64 content from file b64_read.
Sign the manifest with string encode_sign so the partner can check who sent it.
Send the zip with sftp put and the manifest separately.

executed on MentDB

Mainframe fixed-width file to CSV with implied decimals

A mainframe export uses fixed-width records: account on 8 positions, name on 20, amount on 11 digits with 2 implied decimals and a sign, date as YYMMDD. Convert it to a clean CSV and check the control total of the trailer record.

Pipeline

01 · readHeader, details, trailerfile reader_get_line→02 · cutFields by positionstring substring→03 · convertImplied decimals, datesdate format→04 · controlTrailer control totalstring lpad

Script

admin

#1. The export: H header, D details, T trailer with count and total in cents;
-> "[l1]" (concat "D" "00012345" (string rpad "GARAGE HOARAU" " " 20) "+00000124050" "261028");
-> "[l2]" (concat "D" "00012346" (string rpad "PHARMACIE DU PORT" " " 20) "-00000008990" "261029");
-> "[l3]" (concat "D" "00012399" (string rpad "BOULANGERIE PAYET" " " 20) "+00000038520" "261030");
file create "tmp/rx_p4i_mf.dat" (concat "H" "PAYMENTS" "261031" [_n_] [l1] [_n_] [l2] [_n_] [l3] [_n_] "T" (string lpad "3" "0" 6) (string lpad "153580" "0" 13)) "UTF-8";
#2. Read and cut every record by position;
file reader_open "rx_p4i_mf" "tmp/rx_p4i_mf.dat" TEXT "UTF-8";
-> "[csv]" "account;name;amount;date";
-> "[sum]" 0;
-> "[n]" 0;
-> "[rec]" (file reader_get_line "rx_p4i_mf");
while (not (is null [rec])) {
	-> "[type]" (string substring [rec] 0 1);
	if (equal [type] "D") {
		-> "[acc]" (string substring [rec] 1 9);
		-> "[name]" (string capitalize_words (string trim (string substring [rec] 9 29)));
		#3. Sign, 11 digits, 2 implied decimals, date YYMMDD;
		-> "[cents]" (* (if (equal (string substring [rec] 29 30) "-") {-1} {1}) (string substring [rec] 30 41));
		-> "[d]" (date format (concat "20" (string substring [rec] 41 47)) "yyyyMMdd" "yyyy-MM-dd");
		-> "[csv]" (concat [csv] [_n_] [acc] ";" [name] ";" (math decimal_format (/ [cents] 100) "0.00" "." ",") ";" [d]);
		-> "[sum]" (+ [sum] [cents]);
		++ "[n]";
	};
	if (equal [type] "T") {
		-> "[t_count]" (string substring [rec] 1 7);
		-> "[t_total]" (string substring [rec] 7 20);
	};
	-> "[rec]" (file reader_get_line "rx_p4i_mf");
};
file reader_close "rx_p4i_mf";
#4. Control with the trailer: record count and net total in cents;
file create "tmp/rx_p4i_mf.csv" [csv] "UTF-8";
concat [csv] [_n_] "Records read: " [n] ", trailer count: " (math round [t_count] 0) [_n_] "Net total: " (math decimal_format (/ [sum] 100) "0.00" "." ",") " EUR, trailer: " [t_total] ", control ok: " (and (== [n] [t_count]) (equal (string lpad [sum] "0" 13) [t_total]));

Result

mentdb

account;name;amount;date
00012345;Garage Hoarau;1240.50;2026-10-28
00012346;Pharmacie Du Port;-89.90;2026-10-29
00012399;Boulangerie Payet;385.20;2026-10-30
Records read: 3, trailer count: 3
Net total: 1535.80 EUR, trailer: 0000000153580, control ok: 1

Go further