U++ forum: Welcome to the forum

Search on this site

Search in forums

Home » U++ Library support » U++ MT-multithreading and servers » httprequest fails getting entire url content

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

httprequest fails getting entire url content [message #57711]

Wed, 01 December 2021 14:57

aquilarubra
Messages: 23
Registered: December 2019

Promising Member

Hi,
I am experiencing a strange behavior with HttpRequest. It has to get a json page through authorization. It works through a browser, but with HttpRequest it only gets an initial part, then truncates the output with "ERROR: Unexpected end of text."

Any idea what could go wrong?
From the inspector in Firefox I see that only one URL is called. So, no redirections, etc.

Report message to a moderator

Re: httprequest fails getting entire url content [message #57712 is a reply to message #57711]

Wed, 01 December 2021 15:41

Xemuth
Messages: 387
Registered: August 2018
Location: France

Senior Member

Hello Aquilarubra,

What you mean by authorization ? is it a redirection to another website to retrieve a special cookie or something before being sent back to the page ?
I'm curious, I experienced same thing and didn't dig mutch more to fix the problem.

Report message to a moderator

Re: httprequest fails getting entire url content [message #57716 is a reply to message #57712]

Wed, 01 December 2021 20:24

aquilarubra
Messages: 23
Registered: December 2019

Promising Member

I use the authorization() method of HttpRequest. It basically passes an auth user and password encoded in base64. It modifies the header. I said that to give the complete picture, but I don't think it's related yo the issue.
It is like some character is interfering, or HttpRequest foes not wait long enough until the whole content is downloaded. Maybe the called url passes a wrong content size.

The page gives in json or xml format a number of informations. After some general introductory tags, it starts a new tag and should loop yo give out records. HttpRequest always stops with that tag name.

Report message to a moderator

Re: httprequest fails getting entire url content [message #57720 is a reply to message #57716]

Thu, 02 December 2021 09:37

aquilarubra
Messages: 23
Registered: December 2019

Promising Member

I'm trying to debug.

I see that the source is something like this:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?><result><metadata><count>1</count><limit>20</limit><offset>0</offset></metadata><smsList><sms>...

But I get this in HttpResponse:

{"metadata":{"count":1,"offset":0,"limit":20},"smsList":ERROR: Unexpected end of text.

So, the xml parser is engaged by HttpResponse. I see that the error "Unexpected end of text" is in uppsrc/ide/Designers/Xml.cpp, uppsrc/ide/Xml.cpp, and uppsrc/RichText/ParseQtf.cpp

I tried with request.ContentType("text/plain"), but it is still parsed. Is there any way to bypass that behavior?

Report message to a moderator

Re: httprequest fails getting entire url content [message #57721 is a reply to message #57711]

Thu, 02 December 2021 09:58

aquilarubra
Messages: 23
Registered: December 2019

Promising Member

I got the xml not parsed with this, copied from Firefox' Inspector:

request.Accept("text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8");

However, I think that HttpRequest must be reviewed.

Report message to a moderator

Re: httprequest fails getting entire url content [message #57820 is a reply to message #57721]

Sun, 19 December 2021 15:06

mirek
Messages: 13975
Registered: November 2005

Ultimate Member

aquilarubra wrote on Thu, 02 December 2021 09:58

I got the xml not parsed with this, copied from Firefox' Inspector:
request.Accept("text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8");
However, I think that HttpRequest must be reviewed.

Is it possible to have url that is causing this problem?

If not, have you checked with browser that the server is serving the complete content? (Probably yes, but it is always good to check).

Mirek

Report message to a moderator

Re: httprequest fails getting entire url content [message #57821 is a reply to message #57720]

Sun, 19 December 2021 15:10

mirek
Messages: 13975
Registered: November 2005

Ultimate Member

aquilarubra wrote on Thu, 02 December 2021 09:37

I'm trying to debug.

I see that the source is something like this:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?><result><metadata><count>1</count><limit>20</limit><offset>0</offset></metadata><smsList><sms>...
But I get this in HttpResponse:
{"metadata":{"count":1,"offset":0,"limit":20},"smsList":ERROR: Unexpected end of text.
So, the xml parser is engaged by HttpResponse. I see that the error "Unexpected end of text" is in uppsrc/ide/Designers/Xml.cpp, uppsrc/ide/Xml.cpp, and uppsrc/RichText/ParseQtf.cpp

I tried with request.ContentType("text/plain"), but it is still parsed. Is there any way to bypass that behavior?

Ah, this looks like server returns either JSON or XML, based on Accept, defaulting to JSON unless you specify you want XML.

I think .Accept("application/xml") should fix the problem (no need for the long text).

Alternatively, you can simply use JSON (but probably specify .Accept("application/json") just to be futureproof) and ParseJSON instead. JSON is better format anyway... Smile

[Updated on: Sun, 19 December 2021 15:10]

Report message to a moderator

Previous Topic:	Segfault while using BiVector
Next Topic:	Dealing with background tasks elegantly in a userinterface

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Sat Apr 20 10:57:53 CEST 2024

Total time taken to generate the page: 0.05282 seconds