Commit Graph

293 Commits

Author SHA1 Message Date
Nameless 9017d758b9 std.http: use larger read buffer to hit faster tls code 2023-05-06 21:35:17 -05:00
Nameless 1b3ebfefd8 fix keepalive and large buffered writes 2023-05-06 21:35:16 -05:00
Nameless 5f219a2d11 std.http.Server: give Response access to their own allocator
* This makes it easier for threaded servers to use a different allocator
  for each request.
2023-05-06 21:35:16 -05:00
Nameless 7b09629388 std.http: buffer writes 2023-05-06 21:35:16 -05:00
Nameless 533049fdd8 std.http.Server: use enum for reset state instead of bool 2023-05-06 21:35:15 -05:00
Nameless 6513eb4696 std.http.Server: use client recommendation for keepalive 2023-05-06 21:35:15 -05:00
Nameless 71c228fe65 std.http: add simple standalone http tests, add state check for http server 2023-05-06 21:35:15 -05:00
Andrew Kelley 125221cce9 std: update to use @memcpy directly 2023-04-28 13:24:43 -07:00
Nameless 7285eedcd2 std.http: do -> wait, fix redirects 2023-04-26 00:02:55 -07:00
Ryo Ota bba90b8863 fix HTTP server to handle a chunked transfer coding request 2023-04-24 15:36:35 -07:00
Andrew Kelley 1d1255b433 Merge pull request #15416 from squeek502/http-response-headers-undefined
std.http: Always initialize `response.headers` in Client.request
2023-04-24 15:15:17 -07:00
Ryan Liptak 0488c3cb52 std.http: Always initialize response.headers in Client.request
Before this change, if a request errored before getting its `response.headers` initialized, then it would attempt to `deinit` `response.headers` which would still be `undefined`. Since all locations that set `response.headers` use the same code, it can just be done upfront in `request` instead.

Closes #15380
2023-04-23 15:33:23 -07:00
Ryan Liptak 1acb3162b7 http.Headers: Add clearAndFree and clearRetainingCapacity 2023-04-23 15:33:05 -07:00
Ryo Ota afebef2465 create std.http.Server.Response.deinit to handle keepalive connections 2023-04-21 10:17:16 +09:00
Ryo Ota 06763c4c8c move the HTTP test to lib/std/http/test.zig 2023-04-21 09:51:23 +09:00
Ryo Ota 39c0c24b56 fix memory leaks and add an HTTP test 2023-04-21 02:35:38 +09:00
Ryo Ota 0f4f607814 fix http client build error 2023-04-20 22:19:00 +09:00
Ryo Ota d80e6ca5a6 std.http: add missing InvalidTrailers to ReadError 2023-04-20 12:35:26 +03:00
Nameless a23c8662b4 std.http: pass Method to request directly, parse trailing headers 2023-04-18 10:28:53 -05:00
Nameless e65cbff94d std.http: use 'Field' to describe an individual header 2023-04-17 19:16:06 -05:00
Nameless 85221b4e97 std.http: curate some Server errors, fix reading chunked bodies 2023-04-17 19:16:01 -05:00
Nameless 134294230a std.http: add Headers 2023-04-17 19:15:55 -05:00
Nameless 96533b1289 std.http: very basic http client proxy 2023-04-17 19:14:48 -05:00
Nameless 2c492064fb std.http: further curate error set, remove last_error 2023-04-17 19:14:48 -05:00
Nameless 038ed32cff add explicit error union for Bundle.rescan and associated functions 2023-04-17 19:14:48 -05:00
Ryo Ota 17af53554e HTTP client and server send "0\r\n\r\n" when chunked encoding 2023-04-17 13:33:25 +03:00
Nameless 7f9a4625fd std.http: reenable protocol read tests, add missing branch in findHeaders end 2023-04-08 09:59:37 -05:00
Nameless ef6d58ed3b std.http: add documentation 2023-04-08 09:59:36 -05:00
Nameless 8250a92544 update package manager to use req.do(), fix chunked trailer reading 2023-04-08 09:59:36 -05:00
Nameless 52c78f4974 fix bugs, waitForCompleteHead -> do, move redirecting to do instead of read
fix for 32bit arches

curate error sets for api facing functions, expose raw errors in client.last_error

fix bugged dependency loop, disable protocol tests (needs mocking)

add separate mutex for bundle rescan
2023-04-08 09:59:36 -05:00
Nameless aecbfa3a1e add buffering to connection instead of the http protocol, to allow passing through upgrades 2023-04-08 09:59:36 -05:00
Nameless 08bdaf3bd6 std.http: add http server
* extract http protocol into protocol.zig, as it is shared between client and server
* coalesce Request and Response back into Client.zig, they don't contain
  any large chunks of code anymore
* http.Server is implemented as basic as possible, a simple example below:

```zig
fn handler(res: *Server.Response) !void {
    while (true) {
        defer res.reset();

        try res.waitForCompleteHead();
        res.headers.transfer_encoding = .{ .content_length = 14 };
        res.headers.connection = res.request.headers.connection;
        try res.sendResponseHead();
        _ = try res.write("Hello, World!\n");

        if (res.connection.closing) break;
    }
}

pub fn main() !void {
    var server = Server.init(std.heap.page_allocator, .{ .reuse_address = true });
    defer server.deinit();

    try server.listen(try net.Address.parseIp("127.0.0.1", 8080));

    while (true) {
        const res = try server.accept(.{ .dynamic = 8192 });

        const thread = try std.Thread.spawn(.{}, handler, .{res});
        thread.detach();
    }
}
```
2023-04-08 09:59:35 -05:00
Frank Denis 9fedecf4ab http.Client: don't prematurely check transfer_{encoding,compression} (#15040)
Common headers in a response are:

    Content-Encoding: gzip
    Transfer-Encoding: chunked

We used to return `HttpHeadersInvalid` if a `Transfer-Encoding` header
was received while the compression was already set.

However, Transfer-Encoding may not include compression. We should
only return an error if we are setting a value that was already set.

Fixes compatibility with a bunch of websites.
2023-03-23 10:05:58 +01:00
Kotaro Inoue 9ecdcb8e30 Fix to use '/' for a empty path (#14884)
Signed-off-by: Kotaro Inoue <k.musaino@gmail.com>
2023-03-14 13:07:25 +02:00
Nameless 524e0cd987 std.http: rework connection pool into its own type 2023-03-09 14:55:31 -06:00
Nameless 634e715504 std.http: split Client's parts into their own files 2023-03-09 14:55:20 -06:00
Nameless 0a4130f364 std.http: handle relative redirects 2023-03-09 14:55:13 -06:00
Nameless fd2f906d1e std.http: handle compressed payloads 2023-03-09 14:54:26 -06:00
Nameless afb26f4e6b std.http: add connection pooling and make keep-alive requests by default 2023-03-09 14:54:23 -06:00
jim price 6ab04b5941 std.os: Allow write functions to return INVAL errors
In Linux when interacting with the virtual file system when writing
in invalid value to a file the OS will return errno 22 (INVAL).

Instead of triggering an unreachable, this change now returns a
newly introduced error.InvalidArgument.
2023-03-06 15:59:18 -05:00
Andrew Kelley d56a65a8c4 std.http.Client: default to lazy root cert scanning
After this change, the system will be inspected for root certificates
only upon the first https request that actually occurs. This makes the
compiler no longer do SSL certificate scanning when running `zig build`
if no network requests are made.
2023-01-17 01:44:56 -05:00
Andrew Kelley 476cbe871a fix build failures on 32-bit arm due to u64/usize coercion 2023-01-11 15:39:49 -08:00
Andrew Kelley c50f65304f std.http.Client: support the Reader interface 2023-01-11 15:39:48 -08:00
Andrew Kelley aa87789c29 std.Uri: make scheme non-optional 2023-01-06 18:52:39 -07:00
Andrew Kelley 646a911c19 std.http.Client: update from old std.Url to new std.Uri 2023-01-06 18:05:37 -07:00
Andrew Kelley 3806091a10 std.http.Client: fix handling of \r\n before next chunk size 2023-01-06 17:53:06 -07:00
Andrew Kelley a7a933d7ee std.http.Client: support transfer-encoding: chunked
closes #14204

In order to add tests for this I need to implement an HTTP server in the
standard library (#910) so that's probably the next thing I'll do.
2023-01-05 19:57:00 -07:00
Andrew Kelley 3055ab7f86 std.http.Client: fail header parsing under more conditions
* when HTTP header continuations are used
 * when content-type or location header occurs more than once
2023-01-05 13:39:17 -07:00
Andrew Kelley ba1e53f116 avoid triggering LLVM bug on MIPS
See #13782
2023-01-05 00:03:59 -07:00
Andrew Kelley 8248fdbbdb std.http.Client: support HTTP redirects
* std.http.Status.Class: add a "nonstandard" enum tag. Instead of
   having `class` return an optional value, it can potentially return
   nonstandard.
 * extract out std.http.Client.Connection from std.http.Client.Request
   - this code abstracts over plain/TLS only
   - this is the type that will potentially be stored in a client's LRU
     connection map
 * introduce two-staged HTTP header parsing
   - API users can rely on a heap-allocated buffer with a maximum limit,
     which defaults to 16 KB, or they can provide a static buffer that
     is borrowed by the Request instance.
   - The entire HTTP header is buffered because there are strings in
     there and they must be accessed later, such as with the case of
     HTTP redirects.
   - When buffering the HTTP header, the parser only looks for the
     \r\n\r\n pattern. Further validation is done later.
   - After the full HTTP header is buffered, it is parsed into
     components such as Content-Length and Location.
 * HTTP redirects are handled, with a maximum redirect count option that
   defaults to 3.
   - Connection: close is always used for now; implementing keep-alive
     connections and an LRU connection pool in std.http.Client is a task
     for another day.

see #2007
2023-01-04 18:37:53 -07:00