hello,
i can not find a solution for getting separately stdout and stderr using subprocess or command in racket.
with subprocess i get nothing and command i find this solution on the internet:
https://www.reddit.com/r/Racket/comments/tlnyg2/withoutputtostring_cannot_caputure_output_to/
this allow to capture both stdout and stderr but i loose some information , i want to have stdout in a string and stderr in another...
Hi, @damien_mattei.
What are you trying to achieve? I ask because I have only used this in practice once before, with an external Python script.
But basically, you get the ports for the subprocess using:
(define-values (sp out in err)
(subprocess #false #false #false
. apply .
PYTHON-PATH script-path script-args))
And then you use those as you would normally use ports. This is an old piece of code, so it is kind of a mess:
; I think input is supposed to be triggered by a command from the user
(define (script-ready input)
(cond [(and (ports-open?) (process-running?))
(match input
['close
; send the close instruction to the script
(displayln close?
in)
(flush-output in)
; close the ports and wait for the process to terminate
(and (close-ports) (subprocess-wait sp))
#;successful-close 'closed]
[((? string?)
. and .
(app string->bytes/utf-8 text-bytes))
; send the number of bytes for the script to read next
(displayln (bytes-length text-bytes)
in)
(flush-output in)
; send the bytes to the script
(write-bytes text-bytes
in)
(flush-output in)
; apply the port-reader procedure
(on-read out input)]
[else
#;unrecognised-input #false])]
[else
(error 'script-ready
(string-append
"encountered unexpectedly closed port or terminated subprocess:\n"
"`in` closed?: ~a, `out` closed?: ~a, `err` closed?: ~a, `sp` status: ~a")
(port-closed? in) (port-closed? out) (port-closed? err)
(subprocess-status sp))]))
I apologize if this is not what you're looking for, though.
Hello Christiaan,
i wanted to /bin/rm -f files of previous computations, and your code ,remind me that me too i'm launching python code after the /bin/rm -f files and it worked in python (admit i did not test the stderr because there is no error perheaps)
The problem was i used -f in /bin/rm that discard ALL errors so it was normal to have nothing in stderr.
i will rewrite all with subprocess , currently the problem is that /bin/rm -f file can not find file because the current path is not fixed.
i have to fix all that and retest, thanks for your help.
For info here is the scratchy code, but do not take it in consideration ,i have to fix all that:
#! /usr/bin/env racket
#lang reader SRFI-105
;; interpolate field caller
;; Damien MATTEI
;; export PATH=/Applications/Racket/bin:$PATH
(module interpolate-field-caller racket
(require Scheme+)
(require setup/dirs)
(display "Scheme+ : interpole_fields") (newline)
(display "interpole-fields : Racket/Scheme+ library search directory: ") (display (get-lib-search-dirs)) (newline)
(display "interpole-fields : Racket/Scheme+ library collection links:" ) (display (current-library-collection-links)) (newline)
;; launcher directory (web server, command line shell ...)
(define cur-dir (current-directory))
(display "interpole-fields : launcher current directory=") (display cur-dir) (newline)
(current-directory (build-path (current-directory) "drive")) ; setting the current directory for interpolation
(define interpol-dir (current-directory))
(display "interpole-fields : directory for interpolation=") (display interpol-dir) (newline)
;; setting Python paths
(define python-path (getenv "PYTHONPATH"))
(display "interpole-fields : PYTHONPATH=") (display python-path) (newline)
(when (not python-path)
(display "interpole-fields : setting PYTHONPATH") (newline)
(putenv "PYTHONPATH" ".:fibo") ;:drive/.:drive/fibo")
{python-path <- (getenv "PYTHONPATH")} ;(set! python-path (getenv "PYTHONPATH"))
(display "interpole-fields : PYTHONPATH=") (display python-path) (newline))
(define (with-all-output-to-string proc)
(call-with-output-string
(lambda (p) (parameterize ([current-output-port p]
[current-error-port p])
(proc)))))
(define str-out-err (with-all-output-to-string (lambda () (system* "/bin/rm" ;;"/bin/rm"
;;"-f"
;;"drve/output/trajectory-near_Mio_B_6000.txt"
"./drive/output/trajectory-near_Mio_rhoe0_6000.txt"
))))
(display "str-out-err=") (display str-out-err) (newline)
;; (define eff-stdout (open-output-string))
;; (define eff-stderr (open-output-string))
;; (define eff-stdin (open-input-string ""))
;; {(effaceur eff-stdout eff-stdin eff-stderr) <- (subprocess #f #f #f
;; "/bin/rm"
;; "-f"
;; "drve/output/trajectory-near_Mio_B_6000.txt"
;; "drive/output/trajectory-near_Mio_rhoe0_6000.txt")}
;; (define-values (effaceur eff-stdout eff-stdin eff-stderr) (subprocess (current-output-port) ; #f
;; (current-input-port) ;#f
;; (current-error-port) ; #f
;; "/bin/rm"
;; "-f"
;; "drve/output/trajectory-near_Mio_B_6000.txt"
;; "drive/output/trajectory-near_Mio_rhoe0_6000.txt"))
;; (subprocess-wait effaceur)
;; (define eff-err-lns (port->lines eff-stderr))
;; (define eff-len-err-lns (length eff-err-lns))
;; (display "number of lines in eff-stderr=") (display eff-len-err-lns) (newline)
;; (when {eff-len-err-lns > 0}
;; (display "Effaceur Error lines:") (display eff-err-lns) (newline))
;; (define eff-out-lns (port->lines eff-stdout))
;; (display "Output of effaceur:") (display eff-out-lns) (newline)
(exit 0)
(define-values (subby stdout stdin stderr) (subprocess #f #f #f
;;"/usr/bin/env"
"/Library/Frameworks/Python.framework/Versions/3.11/bin/python3.11"
"cut_1D.py"
"../Data" ;"data"
"BepiColombo-Mio_MSO-orbit_1min_short.txt"
"output"))
(display "subby=") (display subby) (newline)
;;(subprocess-wait subby)
(define err-lns (port->lines stderr))
(define len-err-lns (length err-lns))
(display "number of lines in stderr=") (display len-err-lns) (newline)
(when {len-err-lns > 0}
(display "Error lines:") (display err-lns) (newline))
(define out-lns (port->lines stdout))
(display "Output of python interpolation code:") (display out-lns) (newline)
;; for my program the outport port of the subprocess are my input ports and vice versa
(close-input-port stdout)
(close-output-port stdin)
(close-input-port stderr)
) ; end module
i think what deceived me is the -f argument that had hidden all the errors.
I wouldn't launch an external process for that; Racket has functions to get the lists of files in directories and delete files. Just use them. See 15.2 Filesystem
subprocess, system, and similar document how they use ports pretty thoroughly, including the use of current-output-port, etc.: 15.4 Processes
You could parameterize those, or use subprocess-returned ports for something more parallel; but for the case you mentioned, I second using Racket’s (more portable) filesystem constructs.
thanks for the help, both subprocess and process* works now, for deleting i will use filesystems when i update the code. Do not know what was really blocking the process, suddenly it worked all the ways: Racket code is between PHP and Python, and the PHP code is ugly and Python is slow (have to add ini_set('default_socket_timeout', 600); in the PHP soap client too)
(define prcss-lst (process* "/bin/rm"
;;"-f"
"output/trajectory-near_Mio_B_6000.txt"
"output/trajectory-near_Mio_rhoe0_6000.txt"
#:set-pwd? interpol-dir))
(display "interpole_fields : prcss-lst =") (display prcss-lst) (newline)
(define prcss-vct (list->vector prcss-lst))
{eff-stdout <- prcss-vct[0]}
{eff-stdin <- prcss-vct[1]}
{eff-id <- prcss-vct[2]}
{eff-stderr <- prcss-vct[3]}
{eff-proc-ctrl <- prcss-vct[4]}
;;(display "interpole_fields : passed") (newline)
(define eff-err-lns (port->lines eff-stderr))
(define eff-len-err-lns (length eff-err-lns))
(display "number of lines in eff-stderr=") (display eff-len-err-lns) (newline)
(when {eff-len-err-lns > 0}
(display "Effaceur Error lines:") (display eff-err-lns) (newline))
(define eff-out-lns (port->lines eff-stdout))
(display "Output of effaceur:") (display eff-out-lns) (newline)
;; (define-values (subby stdout stdin stderr) (subprocess #f #f #f
;; ;;"/usr/bin/env"
;; ;;"/bin/ls"
;; "/Library/Frameworks/Python.framework/Versions/3.11/bin/python3.11"
;; "cut_1D.py"
;; "../Data"
;; "BepiColombo-Mio_MSO-orbit_1min_short.txt"
;; "output"))
;; ;;(display "subby=") (display subby) (newline)
;; (subprocess-wait subby)
(define prcss-python-lst (process* "/Library/Frameworks/Python.framework/Versions/3.11/bin/python3.11"
"cut_1D.py"
"../Data"
"BepiColombo-Mio_MSO-orbit_1min_short.txt"
"output"
#:set-pwd? interpol-dir))
(display "interpole_fields : prcss-python-lst =") (display prcss-python-lst) (newline)
(define prcss-python-vct (list->vector prcss-python-lst))
{stdout <- prcss-python-vct[0]}
{stdin <- prcss-python-vct[1]}
{python-id <- prcss-python-vct[2]}
{stderr <- prcss-python-vct[3]}
{python-proc-ctrl <- prcss-python-vct[4]}
(define err-lns (port->lines stderr))
(define len-err-lns (length err-lns))
(display "number of lines in stderr=") (display len-err-lns) (newline)
(when {len-err-lns > 0}
(display "Error lines:") (display err-lns) (newline))
(define out-lns (port->lines stdout))
;; (display "Output of python interpolation code:") (display out-lns) (newline)
(display "End of execution of Scheme and Python code.") (newline)
;; for my program the outport port of the subprocess are my input ports and vice versa
(close-input-port stdout)
(close-output-port stdin)
(close-input-port stderr)
the alternative solution commented works too:
(define-values (subby stdout stdin stderr) (subprocess #f #f #f
"/Library/Frameworks/Python.framework/Versions/3.11/bin/python3.11"
"cut_1D.py"
"../Data"
"BepiColombo-Mio_MSO-orbit_1min_short.txt"
; "output"))
;;(display "subby=") (display subby) (newline)
(subprocess-wait subby)
cool to code in Racket but there's a life outside the screens
using 15.2 Filesystem i cannot create a directory, first i use make-directory and now i tried make-directory*, i just got a file with zero bytes:
-rw------- 1 _www wheel 0 18 nov 17:11 VLeG3si
-rw------- 1 _www wheel 0 18 nov 17:11 VLeG3si
drwxrwxrwt 146 root wheel 4672 18 nov 17:11 .
-rwxrw-rw- 1 _www wheel 1313 18 nov 17:11 VLeG3si.xml
-rwxrw-rw- 1 _www wheel 76304 18 nov 17:11 VLeG3si.txt
i used :
(display "interpole-fields : trajectory-txt=") (display trajectory-txt) (newline)
;; creating a temporary directory for output of computation (physical data value at trajectory points)
(make-directory* trajectory-txt)
output log of web server show:
[191] => interpole-fields : trajectory-txt=/private/var/tmp/VLeG3si
[192] => interpole-fields : trajectory-file=/private/var/tmp/VLeG3si.txt
i have regular access to the directory the file could be created but not the directory.
web server run under _www id on Mac OS Sequoia
_www@MacBook-Pro-Touch-Bar tmp % groups
_www wheel everyone staff localaccounts admin com.apple.sharepoint.group.5 _appstore _lpadmin _lpoperator _developer _analyticsusers com.apple.access_ftp com.apple.access_screensharing com.apple.access_ssh-disabled com.apple.access_remote_ae com.apple.sharepoint.group.3 com.apple.sharepoint.group.1 com.apple.sharepoint.group.2 com.apple.sharepoint.group.4
does i used the bad method?
update:
if i test from gui it works:
> (current-directory)
#<path:/Users/mattei/>
> (make-directory* "toto")
> (make-directory* "/private/var/tmp/toto")
-rw------- 1 _www wheel 0 18 nov 17:11 VLeG3si
-rwxrw-rw- 1 _www wheel 1313 18 nov 17:11 VLeG3si.xml
-rwxrw-rw- 1 _www wheel 76304 18 nov 17:11 VLeG3si.txt
drwxr-xr-x@ 2 mattei wheel 64 18 nov 19:37 toto
drwxrwxrwt 147 root wheel 4704 18 nov 19:37 .
if i try from the user _www in command line Racket:
_www@MacBook-Pro-Touch-Bar tmp % who am I
_www ttys004 15 nov 15:42 (::1)
_www@MacBook-Pro-Touch-Bar tmp % export PATH=/Applications/Racket/bin:$PATH
_www@MacBook-Pro-Touch-Bar tmp % racket
Welcome to Racket v8.14 [cs].
> (make-directory* "/private/var/tmp/directory")
> (exit)
_www@MacBook-Pro-Touch-Bar tmp % pwd
/private/var/tmp
_www@MacBook-Pro-Touch-Bar tmp % ls -la directory
total 0
drwxr-xr-x 2 _www wheel 64 18 nov 19:44 .
drwxrwxrwt 148 root wheel 4736 18 nov 19:44 ..
it works too , so it is not a problem of Unix user rights, but why from Racket started from PHP i can not do it? but i can lauch command, python code,etc from same Racket code.
finally, nothing to see with Racket, the PHP parent process was locking the file with : PHP: tempnam - Manual
doing nothing with but locking and removing a 0 byte file (it is not me that write the code) when computation is finished.
$path=tempnam("./tmp","V");
i find that because it worked when modifying the filename in the Scheme part :
{output-dir <- (string-append trajectory-txt "_directory")}
;;(make-directory* trajectory-txt)
(make-directory* output-dir)
-rwxrw-rw- 1 _www wheel 1313 18 nov 20:43 VJZLtSI.xml
-rwxrw-rw- 1 _www wheel 76304 18 nov 20:43 VJZLtSI.txt
drwxr-xr-x@ 2 _www wheel 64 18 nov 20:43 VJZLtSI_directory
note that on Mac OS it is hard to find a unix command that display locked files ,lsof did not help me a lot, happily i got the idea of a deadlock problem.