# Copy-file and upi/cp fail to copy the entire file

**URL:** <https://racket.discourse.group/t/copy-file-and-upi-cp-fail-to-copy-the-entire-file/3406>\
**Category:** Questions & Answers\
**Created:** [December 9, 2024, 2:38pm UTC](https://racket.discourse.group/t/copy-file-and-upi-cp-fail-to-copy-the-entire-file/3406 "2024-12-09T14:38:27Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![damien\_mattei](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/damien_mattei/32/2706_2.png) [@damien\_mattei](https://racket.discourse.group/u/damien_mattei)\
**Post date:** [December 9, 2024, 2:38pm UTC](https://racket.discourse.group/t/copy-file-and-upi-cp-fail-to-copy-the-entire-file/3406/1 "2024-12-09T14:38:27Z")

</div>

hello,

a so weird error i do not expect answers but who knows....

when copying a file with copy-file or upi/cp:

[https://docs.racket-lang.org/upi/upi-cp.html](https://docs.racket-lang.org/upi/upi-cp.html)

it is truncated near the end (496kb file):

```scheme
<TR>
      <TD>
        2021-01-01T23:53:00.000
      </TD>
      <TD>
        -0.479085

```

so i get error in browser :

```scheme
Erreur d’analyse XML : aucun élément trouvé
Emplacement : http://localhost/Vl5OLGf.xml
Numéro de ligne 32998, Colonne 18 :
        -0.479085
-----------------^

```

just wanted to avoid a few lines of code by using copy-file to avoid launching an unix subprocess with cp, so i code it this way:

```scheme
    ;; copy file to make it public to the remote web browser
	{basename-trajectory-xml <- (basename trajectory-xml)}
	;;(copy-file trajectory-xml (string-append "../" basename-trajectory-xml))
	(cp trajectory-xml (string-append "../" basename-trajectory-xml))

```

if i look to the source file on disk the end of file is correct:

```scheme
<TD>
        -0.479085
      </TD>
      <TD>
        0.023290
      </TD>
      <TD>
        -2.295427
      </TD>
      <TD>
        13.800711743772244
      </TD>
      <TD>
        -0.9750889679715302
      </TD>
      <TD>
        13.615658362989326
      </TD>
    </TR>
    <TR>
      <TD>
        2021-01-01T23:54:00.000
      </TD>
      <TD>
        -0.428158
      </TD>
      <TD>
        0.021976
      </TD>
      <TD>
        -2.272187
      </TD>
      <TD>
        14.133451957295375
      </TD>
      <TD>
        -0.896797153024911
      </TD>
      <TD>
        14.160142348754448
      </TD>
    </TR>
    <TR>
      <TD>
        2021-01-01T23:55:00.000
      </TD>
      <TD>
        -0.377049
      </TD>
      <TD>
        0.020649
      </TD>
      <TD>
        -2.247945
      </TD>
      <TD>
        14.281138790035588
      </TD>
      <TD>
        -1.1245551601423487
      </TD>
      <TD>
        14.524911032028472
      </TD>
    </TR>
    <TR>
      <TD>
        2021-01-01T23:56:00.000
      </TD>
      <TD>
        -0.325759
      </TD>
      <TD>
        0.019309
      </TD>
      <TD>
        -2.222663
      </TD>
      <TD>
        14.583629893238435
      </TD>
      <TD>
        -1.208185053380783
      </TD>
      <TD>
        14.87544483985765
      </TD>
    </TR>
    <TR>
      <TD>
        2021-01-01T23:57:00.000
      </TD>
      <TD>
        -0.274306
      </TD>
      <TD>
        0.017956
      </TD>
      <TD>
        -2.196307
      </TD>
      <TD>
        14.818505338078293
      </TD>
      <TD>
        -1.1672597864768683
      </TD>
      <TD>
        15.21708185053381
      </TD>
    </TR>
    <TR>
      <TD>
        2021-01-01T23:58:00.000
      </TD>
      <TD>
        -0.222710
      </TD>
      <TD>
        0.016590
      </TD>
      <TD>
        -2.168841
      </TD>
      <TD>
        15.09608540925267
      </TD>
      <TD>
        -1.2188612099644127
      </TD>
      <TD>
        15.701067615658364
      </TD>
    </TR>
    <TR>
      <TD>
        2021-01-01T23:59:00.000
      </TD>
      <TD>
        -0.170992
      </TD>
      <TD>
        0.015211
      </TD>
      <TD>
        -2.140229
      </TD>
      <TD>
        15.45373665480427
      </TD>
      <TD>
        -1.3558718861209964
      </TD>
      <TD>
        16.144128113879002
      </TD>
    </TR>
  </TABLEDATA>
</DATA>
  </TABLE>
 </RESOURCE>
</VOTABLE>

```

when i look to the end of destination file on disk:

```scheme
      <TD>
        2021-01-01T23:53:00.000
      </TD>
      <TD>
        -0.479085

```

it is truncated. So the copied file is really truncated this is not an effect of network web application or browser.

the source file size is :

499771 bytes, so i do not think it is big or slow to copy it....

i tested with copy-file, upi/cp many times , it does not works...

any idea about what cause that?

update:

here is the file info on disk:

```scheme
ls -la Vh5RkaG.xml /opt/homebrew/var/www/Vh5RkaG.xml 
-rwxrw-rw- 1 _www admin 497382 9 déc 15:27 /opt/homebrew/var/www/Vh5RkaG.xml
-rwxrw-rw- 1 _www wheel 499771 9 déc 15:27 Vh5RkaG.xml

```

clearly it is truncated.

update 2:

but i can copy it manually in command line, just to test if there was some `quota` or something else activate n the filesystem:

```scheme
_www@mbp-touch-bar-1 tmp % cp Vh5RkaG.xml /opt/homebrew/var/www/test.xml   
_www@mbp-touch-bar-1 tmp % ls -la Vh5RkaG.xml /opt/homebrew/var/www/test.xml
-rwxr--r-- 1 _www admin 499771 9 déc 15:47 /opt/homebrew/var/www/test.xml
-rwxrw-rw- 1 _www wheel 499771 9 déc 15:27 Vh5RkaG.xml

```

---

<div class="post-metadata">

**Author:** ![mflatt](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/mflatt/32/6_2.png) [@mflatt](https://racket.discourse.group/u/mflatt)\
**Post date:** [December 9, 2024, 8:28pm UTC](https://racket.discourse.group/t/copy-file-and-upi-cp-fail-to-copy-the-entire-file/3406/2 "2024-12-09T20:28:30Z")

</div>

That does seem strange.

I can imaging getting that result by having `copy-file` called in a thread, and either exiting the process or terminating the thread before it completes, since `copy-file` is not atomic. But probably you're not calling `copy-file` in a thread — right?

Is the truncation always at the same file size or always by the same amount? The `copy-file` function copies in increments of (up to) 4096 bytes, and the file-size delta is less than that in the example, but that may not mean anything.

---

<div class="post-metadata">

**Author:** ![damien\_mattei](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/damien_mattei/32/2706_2.png) [@damien\_mattei](https://racket.discourse.group/u/damien_mattei)\
**Post date:** [December 9, 2024, 9:55pm UTC](https://racket.discourse.group/t/copy-file-and-upi-cp-fail-to-copy-the-entire-file/3406/3 "2024-12-09T21:55:01Z")

</div>

> [@mflatt](#):
>
> I can imaging getting that result by having `copy-file` called in a thread, and either exiting the process or terminating the thread before it completes, since `copy-file` is not atomic. But probably you're not calling `copy-file` in a thread — right?

i did not create a thread but this is a web app, the browser ,request a PHP method that call a web service, here on the same host, the web service is returned by a httpd/php server that launch PHP web service on the server , PHP backend calls Racket program that do some stuff on text and xml files ,etc, remove previous computed file (with rm in a subprocess) and call a Python interpolation code (4-5 minutes) and after Racket build an XML file, this is this XML file that in the last lines of Racket code should copy it on a public directory of the web server to make the remote browser possible access to the resulting XML file.(the file names are always differents in tmp dir)

I did not think there is something cutting the Racket call because it reach the end of the Racket code successfully (see log).

here is the end of Racket code:

```scheme
(display "interpole_fields : end of post-processing XML file: ") (display (current-date)) (newline)

	
	(display "interpole_fields : trajectory-xml=") (display trajectory-xml) (newline)
	
	;; copy file to make it public to the remote web browser
	{basename-trajectory-xml <- (basename trajectory-xml)}
	(copy-file trajectory-xml (string-append "../" basename-trajectory-xml))
	;;(cp trajectory-xml (string-append "../" basename-trajectory-xml))
	
	(display " interpole_fields : last-text=") (display last-text) ; this last line should be in $message variable of PHP caller
       
	
	;; for my program the outport port of the subprocess are my input ports and vice versa
	(close-input-port stdout)
	(close-output-port stdin)
	(close-input-port stderr)

) ; end module

```

and here is the http log:

```scheme
  [24] => interpole_fields : end of post-processing XML file: #(struct:date* 5 27 15 9 12 2024 1 343 #f 3600 216079235 CET)
    [25] => interpole_fields : trajectory-xml=/private/var/tmp/Vh5RkaG.xml
    [26] => interpole_fields : last-text=CPU finished interpolation computation
)

GeneralReturn : this->url_output=/private/var/tmp/Vh5RkaG.xml

```

note: 24,25,26 are lines from Racket code displayed by PHP, GeneralReturn is from PHP code.

So all is running well, no hangup , or crash...

> [@mflatt](#):
>
> Is the truncation always at the same file size or always by the same amount? The `copy-file` function copies in increments of (up to) 4096 bytes, and the file-size delta is less than that in the example, but that may not mean anything.

always at the same point, and strangely near the end as you noticed it, both when using copy-file or upi/cp.

 ![Capture d’écran 2024-12-09 à 22.39.37](https://global.discourse-cdn.com/free1/uploads/racket/original/2X/c/c52ccfc0b8dfd50ec5953cc04a483787572d47d2.png)

Tonight i made a last test from Racket GUI REPL to test `copy-file`and all is good here:

```scheme
Welcome to DrRacket, version 8.14 [cs].
Language: racket, with debugging; memory limit: 8192 MB.
> copy-file
#<procedure:copy-file>
> (copy-file "/private/var/tmp/Vh5RkaG.xml" "/opt/homebrew/var/www/test-from-racket-gui.xml")

```

from the Racket GUI REPL the file are well copied:

```scheme
_www@mbp-touch-bar-1 tmp % ls -la Vh5RkaG.xml /opt/homebrew/var/www/test-from-racket-gui.xml 
-rwxrw-rw-@ 1 mattei admin 499771 9 déc 22:03 /opt/homebrew/var/www/test-from-racket-gui.xml
-rwxrw-rw- 1 _www wheel 499771 9 déc 15:27 Vh5RkaG.xml

```

probably tomorrow i will launch a sub-process with a real `cp` command (as i did it for the `rm` of previous files before using Racket `filesystem` library`) to do the job. Hoping then it will work.

Another last test to see if there is some problem caused by my scheme+ library used in code or the SRFI 105 infix reader, here is a test again in a REPL for scheme+ and SRFI105:

```scheme
Welcome to DrRacket, version 8.14 [cs].
Language: reader SRFI-105, with debugging; memory limit: 8192 MB.
Parsed curly infix code result = 

(module repl racket (provide (all-defined-out)) (require Scheme+))
>
> copy-file

copy-file
#<procedure:copy-file>

#<eof>
> (copy-file "/private/var/tmp/VK6D8Oa.xml" "/opt/homebrew/var/www/test-from-racket-gui-scheme+-reader-srfi105.xml")

(copy-file
 "/private/var/tmp/VK6D8Oa.xml"
 "/opt/homebrew/var/www/test-from-racket-gui-scheme+-reader-srfi105.xml")

#<eof>

```

and the result on disk is ok:

```scheme
_www@mbp-touch-bar-1 tmp % ls -la VK6D8Oa.xml /opt/homebrew/var/www/test-from-racket-gui-scheme+-reader-srfi105.xml 
-rwxrw-rw-@ 1 mattei admin 499771 9 déc 22:49 /opt/homebrew/var/www/test-from-racket-gui-scheme+-reader-srfi105.xml
-rwxrw-rw- 1 _www wheel 499771 9 déc 22:35 VK6D8Oa.xml

```

but the new copied file is again truncated by `copy-file`in the web application at exactly the same byte (size):

```scheme
_www@mbp-touch-bar-1 tmp % ls -la /opt/homebrew/var/www/VK6D8Oa.xml
-rwxrw-rw- 1 _www admin 497382 9 déc 22:35 /opt/homebrew/var/www/VK6D8Oa.xml

```

---

<div class="post-metadata">

**Author:** ![damien\_mattei](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/damien_mattei/32/2706_2.png) [@damien\_mattei](https://racket.discourse.group/u/damien_mattei)\
**Post date:** [December 10, 2024, 11:16am UTC](https://racket.discourse.group/t/copy-file-and-upi-cp-fail-to-copy-the-entire-file/3406/4 "2024-12-10T11:16:39Z")

</div>

more and more strange : even when launching `cp` the file is truncated 😱

```scheme
;; copy it by cp UNIX command
	(define web-root "/opt/homebrew/var/www")
	(define cp-prcss-lst (process* "/bin/cp"
					"-p"
					trajectory-xml
					web-root))

	(display "interpole_fields : cp-prcss-lst =") (display cp-prcss-lst) (newline)
	(define cp-prcss-vct (list->vector cp-prcss-lst))
	{cp-stdout <- cp-prcss-vct[0]}
	{cp-stdin <- cp-prcss-vct[1]}
	{cp-id <- cp-prcss-vct[2]}
	{cp-stderr <- cp-prcss-vct[3]}
	{cp-proc-ctrl <- cp-prcss-vct[4]}
	
	(display "interpole_fields : passed cp") (newline)
	
	(define cp-err-lns (port->lines cp-stderr))
	(define cp-len-err-lns (length cp-err-lns))
	(display "number of lines in cp-stderr=") (display cp-len-err-lns) (newline)
	(when {cp-len-err-lns > 0}
	      (display "cp Error lines:") (display cp-err-lns) (newline))

	(define cp-out-lns (port->lines cp-stdout))
	(display "Output of cp:") (display cp-out-lns) (newline)
	
	(display " interpole_fields : last-text=") (display last-text) ; this last line should be in $message variable of PHP caller
       
	
	;; for my program the outport port of the subprocess are my input ports and vice versa
	(close-input-port stdout)
	(close-output-port stdin)
	(close-input-port stderr)

) ; end module

```

browser:

```scheme
Erreur d’analyse XML : aucun élément trouvé
Emplacement : http://localhost/VK6D8Oa.xml
Numéro de ligne 32998, Colonne 18 :
        -0.479085
-----------------^

```

logs are good:

```scheme
 [25] => interpole_fields : trajectory-xml=/private/var/tmp/VYFfEE0.xml
    [26] => interpole_fields : cp-prcss-lst =(#<input-port:subprocess-stdout> #<output-port:subprocess-stdin> 98850 #<input-port:subprocess-stderr> #<procedure:control>)
    [27] => interpole_fields : passed cp
    [28] => number of lines in cp-stderr=0
    [29] => Output of cp:()
    [30] => interpole_fields : last-text=CPU finished interpolation computation
)

GeneralReturn : this->url_output=/private/var/tmp/VYFfEE0.xml

```

ok... writing that i understand now , if 3 methods failed the problem is in other place.

it is a port of code from [Latmos laboratory](https://latmos.ipsl.fr/en/) (i'm in [Lagrange lab.](https://www.oca.eu/en/welcome-lagrange))

the Latmos code was in PHP and Fortran , our port keeps the main logic of PHP but use Python and Racket instead of Fortran.

the PHP code write the XML header, their fortran code compute but i have not the source code so i do not know where they write the XML body of data, i use the compute code in Python (code written by a student) ,then it generates the XML body data table in Racket and there was a GetURL.php post process at latmos they forget to give to us that should just made public to the world the xml computed locally. I do it in Racket but file was probably close in PHP missing code with perheaps the closing xml tag added unless it was the Fortran code that did it but i will not rely on that, it is what i do me in Racket code but as the XML file was open initially by PHP process i did not close it (this is the PHP missing code job) and the `copy` is made on a non closed,non flushed file and we miss the buffered end of output xml file in the copy.

solution : just add one of those lines:

`(flush-output xml-file)`

or

`(close-output-port xml-file)`

🤦‍♂️ (should have think before)

and it works with any of the 3 methods (copy-file,upi/cp,UNIX cp) :

 ![Capture d’écran 2024-12-10 à 12.09.38](https://global.discourse-cdn.com/free1/uploads/racket/original/2X/9/9c0839c5faf4c9c3f6928a3db4216b4777922ef7.jpeg)