# When there is &quot; in XML, how to get the whole string?

**URL:** <https://racket.discourse.group/t/when-there-is-quot-in-xml-how-to-get-the-whole-string/3144>\
**Category:** Questions & Answers\
**Tags:** racket\
**Created:** [September 2, 2024, 3:50am UTC](https://racket.discourse.group/t/when-there-is-quot-in-xml-how-to-get-the-whole-string/3144 "2024-09-02T03:50:20Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![luistung](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/luistung/32/1865_2.png) [@luistung](https://racket.discourse.group/u/luistung)\
**Post date:** [September 2, 2024, 3:50am UTC](https://racket.discourse.group/t/when-there-is-quot-in-xml-how-to-get-the-whole-string/3144/1 "2024-09-02T03:50:20Z")

</div>

I have a xml file.

```xml
<sst>
    <si><t>{&quot;key1&quot;:&quot;value1&quot;}</t></si>
    <si><t>{"key2":"value2"}</t></si>
</sst>

```

read it using

```lisp
(print
 (xml->xexpr
  (document-element
   (read-xml (open-input-file "problem.xml")))))

```

I got the output

```plaintext
'(sst () "\r\n " (si () (t () "{" "\"" "key1" "\"" ":" "\"" "value1" "\"" "}")) "\r\n " (si () (t () "{\"key2\":\"value2\"}")) "\r\n")

```

the default way to handle the &quot; in string is split the string multiple items.  
how can I get the value1 like value2 straightly without string-join them.

---

<div class="post-metadata">

**Author:** ![damien\_mattei](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/damien_mattei/32/2706_2.png) [@damien\_mattei](https://racket.discourse.group/u/damien_mattei)\
**Post date:** [September 2, 2024, 8:23am UTC](https://racket.discourse.group/t/when-there-is-quot-in-xml-how-to-get-the-whole-string/3144/2 "2024-09-02T08:23:59Z")

</div>

many years i have not parsed XML in Scheme and it was in Kawa but i will try to reply:  
for value1 you can access by a combination of `car` (`first`) and `cdr` (`rest`)  
for value2 i'm a bit puzzled because i'm not sure it ain't a bug in the XML parser library (i'm was facing that already in Kawa but here it is Racket) as you should have get in my opinion more deep parsing and i had expected to have value2 isolated ,not in a string with other data.  
For now you have the solution as for value1 but you will have to finish by spliting the string. Using regular expression can help on the last point...

---

<div class="post-metadata">

**Author:** ![luistung](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/luistung/32/1865_2.png) [@luistung](https://racket.discourse.group/u/luistung)\
**Post date:** [September 2, 2024, 11:42am UTC](https://racket.discourse.group/t/when-there-is-quot-in-xml-how-to-get-the-whole-string/3144/3 "2024-09-02T11:42:31Z")

</div>

Now I just make do with

```scheme
(string-join (cddr t) "")

```

it is so inconvenient that I add this line every time I parse xml.

---

<div class="post-metadata">

**Author:** ![jbclements](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/jbclements/32/11_2.png) [@jbclements](https://racket.discourse.group/u/jbclements)\
**Post date:** [September 2, 2024, 2:01pm UTC](https://racket.discourse.group/t/when-there-is-quot-in-xml-how-to-get-the-whole-string/3144/4 "2024-09-02T14:01:44Z")

</div>

Wait, can we take a step back? Where is this XML coming from? Looks like there's some quoting that needs to be undone before you parse this as XML.

---

<div class="post-metadata">

**Author:** ![jbclements](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/jbclements/32/11_2.png) [@jbclements](https://racket.discourse.group/u/jbclements)\
**Post date:** [September 2, 2024, 2:04pm UTC](https://racket.discourse.group/t/when-there-is-quot-in-xml-how-to-get-the-whole-string/3144/5 "2024-09-02T14:04:56Z")

</div>

Oh... interesting... No, actually, I do think the unquoting is being handled correctly. Yes, I think that the string-join here is appropriate.

You observe that it's inconvenient to be doing string-join, but I would also point out that it's massively inconvenient to ignore all of the newline-whitespace. The more fundamental problem here is that XML is a markup language, not a data encoding language.

But... it kind of looks like this used to be JSON, before it was XML. JSON is a way way nicer place to start from, is there any chance you can get ahold of the JSON that was maybe used to generate this XML?

---

<div class="post-metadata">

**Author:** ![luistung](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/luistung/32/1865_2.png) [@luistung](https://racket.discourse.group/u/luistung)\
**Post date:** [September 3, 2024, 4:05am UTC](https://racket.discourse.group/t/when-there-is-quot-in-xml-how-to-get-the-whole-string/3144/6 "2024-09-03T04:05:26Z")

</div>

I notice in python(xml.dom.minidom) the whole string can be return straightly.  
I am not saying python is right. Each language has their own choice.

I also notice the two xexprs product different xml

```scheme
(display-xml/content
 (xexpr->xml
  '(a quot "key1" quot)))

(display-xml/content
 (xexpr->xml
  '(a "\"key2\"" )))

```

output:

```scheme
<a>&quot;
  key1&quot;
</a>
<a>
  "key2"
</a>

```

so, It makes sense that the read-xml results are different to tell them apart.

---

<div class="post-metadata">

**Author:** ![damien\_mattei](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/damien_mattei/32/2706_2.png) [@damien\_mattei](https://racket.discourse.group/u/damien_mattei)\
**Post date:** [September 3, 2024, 10:07am UTC](https://racket.discourse.group/t/when-there-is-quot-in-xml-how-to-get-the-whole-string/3144/7 "2024-09-03T10:07:24Z")

</div>

that is absolutely normal.

Just a comment : the use of backslash : `\` in many language is used to disable the interpretation of the next character in the language .

as `"` is used to delimiter strings in many languages if you want this character in a string you have to 'backslash' it before.

And by consequence `xexpr->xml` will convert `"key1"` in `key1` and convert `"\"key2\""` in `"key2"`

---

<div class="post-metadata">

**Author:** ![greghendershott](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/greghendershott/32/98_2.png) [@greghendershott](https://racket.discourse.group/u/greghendershott)\
**Post date:** [September 3, 2024, 12:52pm UTC](https://racket.discourse.group/t/when-there-is-quot-in-xml-how-to-get-the-whole-string/3144/8 "2024-09-03T12:52:18Z")

</div>

> [@luistung](#):
>
> I have a xml file.
> 
> ```scheme
> <sst>
> <si><t>{&quot;key1&quot;:&quot;value1&quot;}</t></si>
> <si><t>{"key2":"value2"}</t></si>
> </sst>
> 
> ```

Similar to what @jbclements said:

This XML file is... interesting.

- JSON style data got wrapped in XML, because reasons.

- Furthermore, the item with `&quot;`s seems like an artifact of something encoding JSON for use in HTML, not really XML?

- And that happened only for one `<t>` item, not the other.

In short, looks like data from the real world. 🙂

You'll probably need some pass where you do a certain amount of checking, cleansing, and normalizing, either before or after `read-xml`. Alas this may grow over time as you discover new varieties of "creative expression".

* * *

As @jbclements mentioned, it would be even better if you could get the data as pure JSON. The Racket `json` module works well, in my experience.

(It would also be fine if the data were pure XML. Something like `<t><key>KEY</key><value>VALUE</value></t>` is verbose but consistent.)

---

<div class="post-metadata">

**Author:** ![luistung](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/luistung/32/1865_2.png) [@luistung](https://racket.discourse.group/u/luistung)\
**Post date:** [September 3, 2024, 2:16pm UTC](https://racket.discourse.group/t/when-there-is-quot-in-xml-how-to-get-the-whole-string/3144/9 "2024-09-03T14:16:57Z")

</div>

thanks!

FYI  
This xml is from unzipped xlsx file. As you know, xlsx is a package of xmls.  
Some cells of a sheet are filled in with json string.

---

<div class="post-metadata">

**Author:** ![damien\_mattei](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/damien_mattei/32/2706_2.png) [@damien\_mattei](https://racket.discourse.group/u/damien_mattei)\
**Post date:** [September 3, 2024, 2:54pm UTC](https://racket.discourse.group/t/when-there-is-quot-in-xml-how-to-get-the-whole-string/3144/10 "2024-09-03T14:54:05Z")

</div>

yes i really have to learn JSON 😥 i did not recognize it in the XML sample, it's melting of XML and JSON...

---

<div class="post-metadata">

**Author:** ![LiberalArtist](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/liberalartist/32/151_2.png) [@LiberalArtist](https://racket.discourse.group/u/LiberalArtist)\
**Post date:** [September 3, 2024, 5:44pm UTC](https://racket.discourse.group/t/when-there-is-quot-in-xml-how-to-get-the-whole-string/3144/11 "2024-09-03T17:44:25Z")

</div>

> [@luistung](#):
>
> I also notice the two xexprs product different xml
> 
> ```scheme
> (display-xml/content
> (xexpr->xml
> '(a quot "key1" quot)))
> 
> (display-xml/content
> (xexpr->xml
> '(a "\"key2\"" )))
> 
> ```
> 
> output:
> 
> ```scheme
> <a>&quot;
> key1&quot;
> </a>
> <a>
> "key2"
> </a>
> 
> ```
> 
> so, It makes sense that the read-xml results are different to tell them apart.

One way to understand why `read-xml` might work that way—though not necessarily whether it should!—is to imagine writing a parser. If you are parsing along in a text context and encounter `&`, it signals the start of an entity reference, so you might reasonably close out the pending parsed string before proceeding to parse the entity reference. When the `;` closes the entity reference, you could recognize that `&quot;` is one of the [predefined entities](https://www.w3.org/TR/REC-xml/#sec-predefined-ent) from the XML standard and helpfully represent it as `"\""` rather than `'quot` before resuming parsing in the text context. These design decisions definitely aren't ideal for every possible purpose—sometimes you might really prefer `'quot`, and often a single string would be more convenient—but they are an understandable balance for a general-purpose library.

> [@greghendershott](#):
>
> You'll probably need some pass where you do a certain amount of checking, cleansing, and normalizing, either before or after `read-xml`. Alas this may grow over time as you discover new varieties of "creative expression".

I've done a lot of XML processing in Racket, and it works great, but you _absolutely will_ need to do post-processing to get the data into a more convenient form to work with, especially if the XML is generated by something out of your control.

As your example highlights, the use of `&quot;` is entirely unnecessary except in attribute values, but it is well-formed, and some real-world encoders generate it, so you have to be prepared to handle it. At the extreme, a document could use numeric character references for every single character, so you need to be prepared to use `integer->char`, and `<![CDATA[...]]>` sections are wrapped in a special struct, even though, at the level of the [XML infoset](https://www.w3.org/TR/xml-infoset/), these are all just interchangeable concrete syntaxes for [character data](https://www.w3.org/TR/REC-xml/#syntax)/[character information items](https://www.w3.org/TR/xml-infoset/#infoitem.character).

You will need to handle whitespace. At a minimum, since the `xml` library isn't a "validating XML processor", it can't help with insignificant whitespace in element content, even if you have a DTD defining it as such. In practice, many uses of XML have semantics for whitespace that can't be expressed by a DTD anyway, like the way that, in HTML, whitespace is collapsed in most (but not all) elements.

Similarly, IMHO the biggest weakness of XML is that there aren't really built-in semantics for any datatype but strings and elements. For a specific application of XML, you may want to handle, say, elements for which the order of the children doesn't matter, or you may want to parse the character data of specific elements or attributes as JSON, numbers, booleans, etc.

But you have to do most, if not all, of these things with any generic XML library to work with some specific application of XML. While there are a few things I might change in my personal ideal XML library, the biggest win is getting from XML to lists, symbols, and strings, which Racket then gives you many fantastic tools to transform as you like.
