|
|
Home » U++ Library support » U++ MT-multithreading and servers » [FIXED]HttpRequest hangs when Chunked response, without trailer, and KeepAlive is set. (patch & TC)
[FIXED]HttpRequest hangs when Chunked response, without trailer, and KeepAlive is set. (patch & TC) [message #46292] |
Fri, 08 April 2016 19:44  |
omari
Messages: 276 Registered: March 2010
|
Experienced Member |
|
|
Hi,
When HttpRequest is in TRAILER phase, he call ReadingHeader().
buf if there is no trailer, the socket contain only two charachter: \r\n.
because ReadingHeader() read at least three characters, il hangs waiting a third one.
Here a Test Case:
HttpRequest r("http://dev.alt.cloudappsportal.com/_api/web/lists");
r.KeepAlive().Execute();
A solution is to check if there is a trailer, and if not, read only \r\n and finish:
in the file Core\Http.cpp
Line : 385
replace :
case TRAILER:
if(ReadingHeader())
break;
header.ParseAdd(data);
Finish();
break;
case FINISHED:
by
case TRAILER:
if(TcpSocket::Peek() == '\r') // if the next line is empty (then no trailer)
{
TcpSocket::Get();
c2 = TcpSocket::Get();
if(c2 != '\n')
HttpError("missing ending CRLF");
Finish();
break;
}
if(ReadingHeader())
break;
header.ParseAdd(data);
Finish();
break;
case FINISHED:
regards
omari.
[Updated on: Sat, 18 March 2017 16:01] Report message to a moderator
|
|
|
|
|
|
|
Re: [BUG]HttpRequest hangs when Chunked response, without trailer, and KeepAlive is set. (patch & TC) [message #46320 is a reply to message #46318] |
Mon, 18 April 2016 17:44   |
 |
mirek
Messages: 14257 Registered: November 2005
|
Ultimate Member |
|
|
omari wrote on Sun, 17 April 2016 22:10the trailer part of http response is optional.
an empty line("\r\n") after the last chunk, mean that there is no trailer at all.
in this case, we should go to the next phase : Finish().
bool HttpRequest::ReadingTrailer()
{
for(;;) {
int c = TcpSocket::Get();
if(c < 0)
return !IsEof();
If there is no trailer (or less than 2 characters at the end of stream), it should return false, right?
I believe that calling Finish in all cases is simply wrong - if there is not trailer, it is possible it does not get read. Now obviously, it perhaps does not matter much, unless you are in KeepAlive mode, where you IMO really need trailer to tell the end of each HTTP request.
Mirek
|
|
|
|
|
Re: [BUG]HttpRequest hangs when Chunked response, without trailer, and KeepAlive is set. (patch & TC) [message #46324 is a reply to message #46292] |
Tue, 19 April 2016 11:36   |
omari
Messages: 276 Registered: March 2010
|
Experienced Member |
|
|
Mirek, you are right,
when ReadingTrailer return true, we do not call Finish().
but the actual implementation of ReadingTrailer fail when a Trailer is present.
it miss the part :
if(data.GetCount() > 3) {
const char *h = data.Last();
if(h[0] == '\n' && (h[-1] == '\r' && h[-2] == '\n' || h[-1] == '\n'))
return false;
}
(as ReadingHeader)
bool HttpRequest::ReadingTrailer()
{
for(;;) {
int c = TcpSocket::Get();
if(c < 0)
return !IsEof();
else
data.Cat(c);
if(data.GetCount() == 2) {
if(data[0] == '\r' && data[1] == '\n')
return false;
}
if(data.GetCount() > 3) {
const char *h = data.Last();
if(h[0] == '\n' && (h[-1] == '\r' && h[-2] == '\n' || h[-1] == '\n'))
return false;
}
}
}
regards
omari.
[Updated on: Tue, 19 April 2016 11:36] Report message to a moderator
|
|
|
Re: [BUG]HttpRequest hangs when Chunked response, without trailer, and KeepAlive is set. (patch & TC) [message #46325 is a reply to message #46292] |
Tue, 19 April 2016 11:42   |
omari
Messages: 276 Registered: March 2010
|
Experienced Member |
|
|
here a test case for chunker response with and without trailer:
#include <CtrlLib/CtrlLib.h>
using namespace Upp;
// base source : http://www.tcpipguide.com/free/t_HTTPDataLengthIssuesChunkedTransfersandMessageTrai-3.htm
String chunked_with_trailer = "HTTP/1.1 200 OK\r\nDate: Mon, 22 Mar 2004 11:15:03 GMT\r\nContent-Type: text/html\r\nTransfer-Encoding: chunked\r\nTrailer: Expires\r\n\r\n29\r\n<html><body><p>The file you requested is \r\n5\r\n3,400\r\n23\r\nbytes long and was last modified:\r\n\r\n1d\r\nSat, 20 Mar 2004 21:12:00 GMT\r\n13\r\n.</p></body></html>\r\n0\r\nExpires: Sat, 27 Mar 2004 21:12:00 GMT\r\n\r\n";
// base source : https://en.wikipedia.org/wiki/Chunked_transfer_encoding?oldid=430331176
String chunked_without_trailer = "HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nTransfer-Encoding: chunked\r\n\r\n26\r\nThis is the data in the first chunk\r\n6\r\n1C\r\nand this is the second one\r\n\r\n3\r\ncon\r\n8\r\nsequence\r\n0\r\n\r\n";
static void Server(String r)
{
TcpSocket server;
if(server.Listen(4000, 10)) {
TcpSocket socket;
LOG("Waiting...");
bool b = socket.Accept(server);
if(b) {
LOG("Connection accepted");
HttpHeader http;
http.Read(socket);
socket.Put(r);
socket.Close();
}
}
}
GUI_APP_MAIN
{
StdLogSetup(LOG_COUT|LOG_FILE);
Thread a;
LOG("chunked_without_trailer");
LOG("****************");
LOG(chunked_without_trailer);
LOG("---------------");
a.Run(callback1(Server, chunked_without_trailer));
HttpRequest r1("localhost:4000");/*r1.Trace();*/ LOG(r1.GET().Execute());
a.Wait();
LOG("chunked_with_trailer");
LOG("****************");
LOG(chunked_with_trailer);
LOG("---------------");
a.Run(callback1(Server, chunked_with_trailer));
HttpRequest r2("localhost:4000");/*r2.Trace();*/ LOG(r2.GET().Execute());
a.Wait();
LOG("=============== OK");
}
regards
omari.
|
|
|
|
Re: [BUG]HttpRequest hangs when Chunked response, without trailer, and KeepAlive is set. (patch & TC) [message #46335 is a reply to message #46332] |
Fri, 22 April 2016 17:08   |
omari
Messages: 276 Registered: March 2010
|
Experienced Member |
|
|
Thanks Mirek, it work well for this scenario.
there are two other small bugs that appears with KeepAlive:
1 - Finish() calls Close() (at line 833), but in keep_alive mode, it should not
2 - there is a blocking, when response has Content-Length = 0, for example:
HTTP/1.0 200 OK
Date: Fri, 31 Dec 1999 23:59:59 GMT
Server: Apache/0.8.4
Content-Type: text/html
Content-Length: 0
Expires: Sat, 01 Jan 2000 00:59:59 GMT
Last-modified: Fri, 09 Aug 1996 14:21:40 GMT
a possible solution (tested) is in ReadingBody()
replace:
if(count > 0)
n = (int)min((int64)n, count);
by
if(count == 0)
return false;
n = (int)min((int64)n, count);
regards
omari.
|
|
|
|
|
Goto Forum:
Current Time: Sat May 10 00:41:35 CEST 2025
Total time taken to generate the page: 0.01288 seconds
|
|
|