dovecot/core-2.2: src/lib-http/http-client-queue.c annotate

annotate src/lib-http/http-client-queue.c @ 21466:10103d3a4ad4

lib-http: client: Fixed peer reconnection failure handling. The addressed problem occurs in a very specific situation in which the original successful connection is dropped, yet a new connection fails. It manifests as an assertion failure or panic: Panic: file ioloop-epoll.c: line 189 (io_loop_handler_run_internal): assertion failed: (msecs >= 0) Panic: BUG: No IOs or timeouts set. Not waiting for infinity. The timing is very critical. However, this doesn't mean that the occurrence of this problem is very unlikely; it can happen frequently under high load.

author	Stephan Bosch <stephan.bosch@dovecot.fi>
date	Thu, 02 Feb 2017 01:36:50 +0100
parents	2e2563132d5f
children	0d071dadb856

rev	line source
21390 2e2563132d5f Updated copyright notices to include the year 2017. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21263 diff changeset	1 /* Copyright (c) 2013-2017 Dovecot authors, see the included COPYING file */
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	2
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	3 #include "lib.h"
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	4 #include "net.h"
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	5 #include "str.h"
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	6 #include "hash.h"
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	7 #include "array.h"
17005 d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	8 #include "bsearch-insert-pos.h"
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	9 #include "llist.h"
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	10 #include "ioloop.h"
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	11 #include "istream.h"
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	12 #include "ostream.h"
17005 d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	13 #include "time-util.h"
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	14 #include "dns-lookup.h"
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	15 #include "http-response-parser.h"
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	16
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	17 #include "http-client-private.h"
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	18
17879 94ba0150905b lib-http: client queue: Start using new timeval_cmp_margin function for delay handling. Stephan Bosch <stephan@rename-it.nl> parents: 17852 diff changeset	19 #define TIMEOUT_CMP_MARGIN_USECS 2000
94ba0150905b lib-http: client queue: Start using new timeval_cmp_margin function for delay handling. Stephan Bosch <stephan@rename-it.nl> parents: 17852 diff changeset	20
17881 4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	21 static void
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	22 http_client_queue_set_delay_timer(struct http_client_queue *queue,
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	23 struct timeval time);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	24 static void
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	25 http_client_queue_set_request_timer(struct http_client_queue *queue,
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	26 const struct timeval *time);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	27
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	28
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	29 /*
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	30 * Logging
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	31 */
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	32
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	33 static inline void
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	34 http_client_queue_debug(struct http_client_queue *queue,
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	35 const char *format, ...) ATTR_FORMAT(2, 3);
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	36
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	37 static inline void
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	38 http_client_queue_debug(struct http_client_queue *queue,
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	39 const char *format, ...)
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	40 {
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	41 va_list args;
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	42
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	43 if (queue->client->set.debug) {
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	44
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	45 va_start(args, format);
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	46 i_debug("http-client: queue %s: %s",
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	47 queue->name, t_strdup_vprintf(format, args));
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	48 va_end(args);
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	49 }
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	50 }
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	51
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	52 /*
17881 4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	53 * Queue object
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	54 */
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	55
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	56 static struct http_client_queue *
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	57 http_client_queue_find(struct http_client_host *host,
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	58 const struct http_client_peer_addr *addr)
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	59 {
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	60 struct http_client_queue const queue_idx;
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	61
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	62 array_foreach_modifiable(&host->queues, queue_idx) {
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	63 struct http_client_queue queue = queue_idx;
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	64
19039 8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	65 if (http_client_peer_addr_cmp(&queue->addr, addr) == 0)
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	66 return queue;
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	67 }
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	68
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	69 return NULL;
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	70 }
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	71
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	72 struct http_client_queue *
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	73 http_client_queue_create(struct http_client_host *host,
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	74 const struct http_client_peer_addr *addr)
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	75 {
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	76 struct http_client_queue *queue;
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	77
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	78 queue = http_client_queue_find(host, addr);
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	79 if (queue == NULL) {
19039 8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	80 queue = i_new(struct http_client_queue, 1);
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	81 queue->client = host->client;
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	82 queue->host = host;
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	83 queue->addr = *addr;
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	84
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	85 switch (addr->type) {
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	86 case HTTP_CLIENT_PEER_ADDR_RAW:
19039 8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	87 queue->name =
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	88 i_strdup_printf("raw://%s:%u", host->name, addr->a.tcp.port);
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	89 queue->addr.a.tcp.https_name = NULL;
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	90 break;
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	91 case HTTP_CLIENT_PEER_ADDR_HTTPS_TUNNEL:
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	92 case HTTP_CLIENT_PEER_ADDR_HTTPS:
19039 8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	93 queue->name =
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	94 i_strdup_printf("https://%s:%u", host->name, addr->a.tcp.port);
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	95 queue->addr_name = i_strdup(addr->a.tcp.https_name);
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	96 queue->addr.a.tcp.https_name = queue->addr_name;
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	97 break;
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	98 case HTTP_CLIENT_PEER_ADDR_HTTP:
19039 8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	99 queue->name =
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	100 i_strdup_printf("http://%s:%u", host->name, addr->a.tcp.port);
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	101 queue->addr.a.tcp.https_name = NULL;
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	102 break;
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	103 case HTTP_CLIENT_PEER_ADDR_UNIX:
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	104 queue->name = i_strdup_printf("unix:%s", addr->a.un.path);
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	105 queue->addr_name = i_strdup(addr->a.un.path);
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	106 queue->addr.a.un.path = queue->addr_name;
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	107 break;
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	108 default:
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	109 i_unreached();
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	110 }
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	111
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	112 queue->ips_connect_idx = 0;
21045 c926a09729f0 lib-http: client: Create queue->pending_peers array from the beginning. Stephan Bosch <stephan@dovecot.fi> parents: 21044 diff changeset	113 i_array_init(&queue->pending_peers, 8);
17881 4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	114 i_array_init(&queue->requests, 16);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	115 i_array_init(&queue->queued_requests, 16);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	116 i_array_init(&queue->queued_urgent_requests, 16);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	117 i_array_init(&queue->delayed_requests, 4);
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	118 array_append(&host->queues, &queue, 1);
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	119 }
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	120
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	121 return queue;
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	122 }
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	123
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	124 void http_client_queue_free(struct http_client_queue *queue)
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	125 {
21044 66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	126 struct http_client_peer const peer_idx;
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	127 ARRAY_TYPE(http_client_peer) peers;
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	128
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	129 http_client_queue_debug(queue, "Destroy");
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	130
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	131 /* unlink all peers */
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	132 if (queue->cur_peer != NULL) {
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	133 struct http_client_peer *peer = queue->cur_peer;
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	134 queue->cur_peer = NULL;
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	135 http_client_peer_unlink_queue(peer, queue);
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	136 }
21045 c926a09729f0 lib-http: client: Create queue->pending_peers array from the beginning. Stephan Bosch <stephan@dovecot.fi> parents: 21044 diff changeset	137 t_array_init(&peers, array_count(&queue->pending_peers));
c926a09729f0 lib-http: client: Create queue->pending_peers array from the beginning. Stephan Bosch <stephan@dovecot.fi> parents: 21044 diff changeset	138 array_copy(&peers.arr, 0, &queue->pending_peers.arr, 0,
c926a09729f0 lib-http: client: Create queue->pending_peers array from the beginning. Stephan Bosch <stephan@dovecot.fi> parents: 21044 diff changeset	139 array_count(&queue->pending_peers));
c926a09729f0 lib-http: client: Create queue->pending_peers array from the beginning. Stephan Bosch <stephan@dovecot.fi> parents: 21044 diff changeset	140 array_foreach(&peers, peer_idx)
c926a09729f0 lib-http: client: Create queue->pending_peers array from the beginning. Stephan Bosch <stephan@dovecot.fi> parents: 21044 diff changeset	141 http_client_peer_unlink_queue(*peer_idx, queue);
c926a09729f0 lib-http: client: Create queue->pending_peers array from the beginning. Stephan Bosch <stephan@dovecot.fi> parents: 21044 diff changeset	142 array_free(&queue->pending_peers);
21044 66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	143
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	144 /* abort all requests */
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	145 http_client_queue_fail
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	146 (queue, HTTP_CLIENT_REQUEST_ERROR_ABORTED, "Aborted");
17881 4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	147 array_free(&queue->requests);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	148 array_free(&queue->queued_requests);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	149 array_free(&queue->queued_urgent_requests);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	150 array_free(&queue->delayed_requests);
21044 66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	151
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	152 /* cancel timeouts */
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	153 if (queue->to_connect != NULL)
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	154 timeout_remove(&queue->to_connect);
17005 d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	155 if (queue->to_delayed != NULL)
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	156 timeout_remove(&queue->to_delayed);
21044 66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	157
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	158 /* free */
19039 8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	159 i_free(queue->addr_name);
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	160 i_free(queue->name);
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	161 i_free(queue);
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	162 }
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	163
17881 4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	164 /*
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	165 * Error handling
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	166 */
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	167
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	168 void http_client_queue_fail(struct http_client_queue *queue,
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	169 unsigned int status, const char *error)
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	170 {
17852 b7678ce561c1 lib-http: client: The http_client_queue_fail() function aborted requests in an unsafe manner. Stephan Bosch <stephan@rename-it.nl> parents: 17851 diff changeset	171 ARRAY_TYPE(http_client_request) *req_arr, treqs;
b7678ce561c1 lib-http: client: The http_client_queue_fail() function aborted requests in an unsafe manner. Stephan Bosch <stephan@rename-it.nl> parents: 17851 diff changeset	172 struct http_client_request **req_idx;
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	173
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	174 /* abort all pending requests */
17881 4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	175 req_arr = &queue->requests;
17852 b7678ce561c1 lib-http: client: The http_client_queue_fail() function aborted requests in an unsafe manner. Stephan Bosch <stephan@rename-it.nl> parents: 17851 diff changeset	176 t_array_init(&treqs, array_count(req_arr));
b7678ce561c1 lib-http: client: The http_client_queue_fail() function aborted requests in an unsafe manner. Stephan Bosch <stephan@rename-it.nl> parents: 17851 diff changeset	177 array_copy(&treqs.arr, 0, &req_arr->arr, 0, array_count(req_arr));
b7678ce561c1 lib-http: client: The http_client_queue_fail() function aborted requests in an unsafe manner. Stephan Bosch <stephan@rename-it.nl> parents: 17851 diff changeset	178 array_foreach_modifiable(&treqs, req_idx) {
20344 2b71a1c2febc lib-http: Changed http_client_request_error to set request to NULL Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20332 diff changeset	179 http_client_request_error(req_idx, status, error);
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	180 }
17005 d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	181
17985 e479bce0633b lib-http: client: Assertion in http_client_queue_fail() was wrong. Stephan Bosch <stephan@rename-it.nl> parents: 17969 diff changeset	182 /* all queues should be empty now... unless new requests were submitted
e479bce0633b lib-http: client: Assertion in http_client_queue_fail() was wrong. Stephan Bosch <stephan@rename-it.nl> parents: 17969 diff changeset	183 from the callback. this invariant captures it all: */
e479bce0633b lib-http: client: Assertion in http_client_queue_fail() was wrong. Stephan Bosch <stephan@rename-it.nl> parents: 17969 diff changeset	184 i_assert((array_count(&queue->delayed_requests) +
e479bce0633b lib-http: client: Assertion in http_client_queue_fail() was wrong. Stephan Bosch <stephan@rename-it.nl> parents: 17969 diff changeset	185 array_count(&queue->queued_requests) +
e479bce0633b lib-http: client: Assertion in http_client_queue_fail() was wrong. Stephan Bosch <stephan@rename-it.nl> parents: 17969 diff changeset	186 array_count(&queue->queued_urgent_requests)) ==
e479bce0633b lib-http: client: Assertion in http_client_queue_fail() was wrong. Stephan Bosch <stephan@rename-it.nl> parents: 17969 diff changeset	187 array_count(&queue->requests));
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	188 }
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	189
17881 4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	190 /*
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	191 * Connection management
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	192 */
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	193
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	194 static bool
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	195 http_client_queue_is_last_connect_ip(struct http_client_queue *queue)
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	196 {
17774 650629de6a73 lib-http: client: Added support for attempting a single IP several times. Stephan Bosch <stephan@rename-it.nl> parents: 17172 diff changeset	197 const struct http_client_settings *set =
650629de6a73 lib-http: client: Added support for attempting a single IP several times. Stephan Bosch <stephan@rename-it.nl> parents: 17172 diff changeset	198 &queue->client->set;
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	199 struct http_client_host *host = queue->host;
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	200
19039 8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	201 i_assert(queue->addr.type != HTTP_CLIENT_PEER_ADDR_UNIX);
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	202 i_assert(queue->ips_connect_idx < host->ips_count);
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	203 i_assert(queue->ips_connect_start_idx < host->ips_count);
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	204
17774 650629de6a73 lib-http: client: Added support for attempting a single IP several times. Stephan Bosch <stephan@rename-it.nl> parents: 17172 diff changeset	205 /* if a maximum connect attempts > 1 is set, enforce it directly */
17942 b254b7dc717f lib-http: client: With the recent addition of a connection attempt limit, connection failures weren't always handled correctly. Stephan Bosch <stephan@rename-it.nl> parents: 17881 diff changeset	206 if (set->max_connect_attempts > 1 &&
b254b7dc717f lib-http: client: With the recent addition of a connection attempt limit, connection failures weren't always handled correctly. Stephan Bosch <stephan@rename-it.nl> parents: 17881 diff changeset	207 queue->connect_attempts >= set->max_connect_attempts)
b254b7dc717f lib-http: client: With the recent addition of a connection attempt limit, connection failures weren't always handled correctly. Stephan Bosch <stephan@rename-it.nl> parents: 17881 diff changeset	208 return TRUE;
17774 650629de6a73 lib-http: client: Added support for attempting a single IP several times. Stephan Bosch <stephan@rename-it.nl> parents: 17172 diff changeset	209
650629de6a73 lib-http: client: Added support for attempting a single IP several times. Stephan Bosch <stephan@rename-it.nl> parents: 17172 diff changeset	210 /* otherwise, we'll always go through all the IPs. we don't necessarily
650629de6a73 lib-http: client: Added support for attempting a single IP several times. Stephan Bosch <stephan@rename-it.nl> parents: 17172 diff changeset	211 start connecting from the first IP, so we'll need to treat the IPs as
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	212 a ring buffer where we automatically wrap back to the first IP
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	213 when necessary. */
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	214 return (queue->ips_connect_idx + 1) % host->ips_count ==
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	215 queue->ips_connect_start_idx;
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	216 }
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	217
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	218 static void
21047 3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	219 http_client_queue_recover_from_lookup(struct http_client_queue *queue)
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	220 {
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	221 struct http_client_host *host = queue->host;
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	222 struct http_client_peer_addr new_addr = queue->addr;
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	223 unsigned int i;
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	224
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	225 i_assert(queue->addr.type != HTTP_CLIENT_PEER_ADDR_UNIX);
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	226
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	227 if (queue->cur_peer == NULL) {
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	228 queue->ips_connect_idx = queue->ips_connect_start_idx = 0;
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	229 return;
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	230 }
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	231
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	232 /* try to find current peer amongst new IPs */
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	233 for (i = 0; i < host->ips_count; i++) {
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	234 new_addr.a.tcp.ip = host->ips[i];
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	235 if (http_client_peer_addr_cmp
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	236 (&new_addr, &queue->cur_peer->addr) == 0)
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	237 break;
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	238 }
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	239
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	240 if (i < host->ips_count) {
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	241 /* continue with current peer */
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	242 queue->ips_connect_idx = queue->ips_connect_start_idx = i;
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	243 } else {
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	244 /* reset connect attempts */
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	245 queue->ips_connect_idx = queue->ips_connect_start_idx = 0;
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	246 }
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	247 }
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	248
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	249 static void
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	250 http_client_queue_soft_connect_timeout(struct http_client_queue *queue)
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	251 {
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	252 struct http_client_host *host = queue->host;
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	253 const struct http_client_peer_addr *addr = &queue->addr;
19039 8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	254 const char *https_name;
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	255
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	256 i_assert(queue->addr.type != HTTP_CLIENT_PEER_ADDR_UNIX);
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	257
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	258 if (queue->to_connect != NULL)
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	259 timeout_remove(&queue->to_connect);
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	260
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	261 if (http_client_queue_is_last_connect_ip(queue)) {
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	262 /* no more IPs to try */
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	263 return;
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	264 }
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	265
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	266 /* if our our previous connection attempt takes longer than the
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	267 soft_connect_timeout, we start a connection attempt to the next IP in
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	268 parallel */
19039 8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	269 https_name = http_client_peer_addr_get_https_name(addr);
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	270 http_client_queue_debug(queue, "Connection to %s%s is taking a long time; "
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	271 "starting parallel connection attempt to next IP",
19039 8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	272 http_client_peer_addr2str(addr), (https_name == NULL ? "" :
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	273 t_strdup_printf(" (SSL=%s)", https_name)));
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	274
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	275 /* next IP */
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	276 queue->ips_connect_idx = (queue->ips_connect_idx + 1) % host->ips_count;
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	277
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	278 /* setup connection to new peer (can start new soft timeout) */
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	279 http_client_queue_connection_setup(queue);
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	280 }
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	281
21044 66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	282 static struct http_client_peer *
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	283 http_client_queue_connection_attempt(struct http_client_queue *queue)
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	284 {
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	285 struct http_client_host *host = queue->host;
21044 66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	286 struct http_client_peer *peer;
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	287 struct http_client_peer_addr *addr = &queue->addr;
17881 4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	288 unsigned int num_requests =
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	289 array_count(&queue->queued_requests) +
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	290 array_count(&queue->queued_urgent_requests);
19039 8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	291 const char *ssl = "";
21047 3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	292 int ret;
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	293
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	294 if (num_requests == 0)
21044 66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	295 return NULL;
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	296
21047 3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	297 /* check whether host IPs are still up-to-date */
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	298 if ((ret=http_client_host_refresh(host)) < 0) {
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	299 /* performing asynchronous lookup */
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	300 if (queue->to_connect != NULL)
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	301 timeout_remove(&queue->to_connect);
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	302 return NULL;
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	303 }
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	304 if (ret > 0) {
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	305 /* new lookup performed */
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	306 http_client_queue_recover_from_lookup(queue);
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	307 }
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	308
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	309 /* update our peer address */
19039 8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	310 if (queue->addr.type != HTTP_CLIENT_PEER_ADDR_UNIX) {
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	311 i_assert(queue->ips_connect_idx < host->ips_count);
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	312 queue->addr.a.tcp.ip = host->ips[queue->ips_connect_idx];
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	313 ssl = http_client_peer_addr_get_https_name(addr);
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	314 ssl = (ssl == NULL ? "" : t_strdup_printf(" (SSL=%s)", ssl));
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	315 }
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	316
21044 66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	317 /* already got a peer? */
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	318 peer = NULL;
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	319 if (queue->cur_peer != NULL) {
21045 c926a09729f0 lib-http: client: Create queue->pending_peers array from the beginning. Stephan Bosch <stephan@dovecot.fi> parents: 21044 diff changeset	320 i_assert(array_count(&queue->pending_peers) == 0);
21044 66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	321
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	322 /* is it still the one we want? */
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	323 if (http_client_peer_addr_cmp(addr, &queue->cur_peer->addr) == 0) {
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	324 /* is it still connected? */
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	325 if (http_client_peer_is_connected(queue->cur_peer)) {
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	326 /* yes */
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	327 http_client_queue_debug(queue,
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	328 "Using existing connection to %s%s "
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	329 "(%u requests pending)",
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	330 http_client_peer_addr2str(addr), ssl, num_requests);
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	331
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	332 /* handle requests; */
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	333 http_client_peer_trigger_request_handler(queue->cur_peer);
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	334 return queue->cur_peer;
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	335 }
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	336 /* no */
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	337 peer = queue->cur_peer;
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	338 } else {
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	339 /* peer is not relevant to this queue anymore */
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	340 http_client_peer_unlink_queue(queue->cur_peer, queue);
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	341 }
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	342
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	343 queue->cur_peer = NULL;
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	344 }
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	345
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	346 if (peer == NULL)
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	347 peer = http_client_peer_get(queue->client, addr);
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	348
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	349 http_client_queue_debug(queue, "Setting up connection to %s%s "
19039 8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	350 "(%u requests pending)", http_client_peer_addr2str(addr), ssl,
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	351 num_requests);
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	352
21044 66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	353 /* create provisional link between queue and peer */
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	354 http_client_peer_link_queue(peer, queue);
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	355
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	356 /* handle requests; creates new connections when needed/possible */
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	357 http_client_peer_trigger_request_handler(peer);
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	358
21044 66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	359 if (http_client_peer_is_connected(peer)) {
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	360 /* drop any pending peers */
21045 c926a09729f0 lib-http: client: Create queue->pending_peers array from the beginning. Stephan Bosch <stephan@dovecot.fi> parents: 21044 diff changeset	361 if (array_count(&queue->pending_peers) > 0) {
21044 66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	362 struct http_client_peer const peer_idx;
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	363
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	364 array_foreach(&queue->pending_peers, peer_idx) {
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	365 i_assert(http_client_peer_addr_cmp(&(*peer_idx)->addr, addr) != 0);
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	366 http_client_peer_unlink_queue(*peer_idx, queue);
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	367 }
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	368 array_clear(&queue->pending_peers);
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	369 }
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	370 queue->cur_peer = peer;
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	371
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	372 } else {
21045 c926a09729f0 lib-http: client: Create queue->pending_peers array from the beginning. Stephan Bosch <stephan@dovecot.fi> parents: 21044 diff changeset	373 struct http_client_peer const peer_idx;
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	374 unsigned int msecs;
17172 57638bf8ee3e lib-http: Fixed a hang Timo Sirainen <tss@iki.fi> parents: 17171 diff changeset	375 bool new_peer = TRUE;
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	376
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	377 /* not already connected, wait for connections */
17172 57638bf8ee3e lib-http: Fixed a hang Timo Sirainen <tss@iki.fi> parents: 17171 diff changeset	378
21045 c926a09729f0 lib-http: client: Create queue->pending_peers array from the beginning. Stephan Bosch <stephan@dovecot.fi> parents: 21044 diff changeset	379 /* we may be waiting for this peer already */
c926a09729f0 lib-http: client: Create queue->pending_peers array from the beginning. Stephan Bosch <stephan@dovecot.fi> parents: 21044 diff changeset	380 array_foreach(&queue->pending_peers, peer_idx) {
c926a09729f0 lib-http: client: Create queue->pending_peers array from the beginning. Stephan Bosch <stephan@dovecot.fi> parents: 21044 diff changeset	381 if (http_client_peer_addr_cmp(&(*peer_idx)->addr, addr) == 0) {
c926a09729f0 lib-http: client: Create queue->pending_peers array from the beginning. Stephan Bosch <stephan@dovecot.fi> parents: 21044 diff changeset	382 i_assert(*peer_idx == peer);
c926a09729f0 lib-http: client: Create queue->pending_peers array from the beginning. Stephan Bosch <stephan@dovecot.fi> parents: 21044 diff changeset	383 new_peer = FALSE;
c926a09729f0 lib-http: client: Create queue->pending_peers array from the beginning. Stephan Bosch <stephan@dovecot.fi> parents: 21044 diff changeset	384 break;
17172 57638bf8ee3e lib-http: Fixed a hang Timo Sirainen <tss@iki.fi> parents: 17171 diff changeset	385 }
57638bf8ee3e lib-http: Fixed a hang Timo Sirainen <tss@iki.fi> parents: 17171 diff changeset	386 }
17774 650629de6a73 lib-http: client: Added support for attempting a single IP several times. Stephan Bosch <stephan@rename-it.nl> parents: 17172 diff changeset	387 if (new_peer) {
650629de6a73 lib-http: client: Added support for attempting a single IP several times. Stephan Bosch <stephan@rename-it.nl> parents: 17172 diff changeset	388 http_client_queue_debug(queue, "Started new connection to %s%s",
19039 8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	389 http_client_peer_addr2str(addr), ssl);
17774 650629de6a73 lib-http: client: Added support for attempting a single IP several times. Stephan Bosch <stephan@rename-it.nl> parents: 17172 diff changeset	390
17172 57638bf8ee3e lib-http: Fixed a hang Timo Sirainen <tss@iki.fi> parents: 17171 diff changeset	391 array_append(&queue->pending_peers, &peer, 1);
20152 b983a3ab3fcf lib-http: If connect fails, include attempt count and total time in error. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 19552 diff changeset	392 if (queue->connect_attempts++ == 0)
b983a3ab3fcf lib-http: If connect fails, include attempt count and total time in error. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 19552 diff changeset	393 queue->first_connect_time = ioloop_timeval;
17774 650629de6a73 lib-http: client: Added support for attempting a single IP several times. Stephan Bosch <stephan@rename-it.nl> parents: 17172 diff changeset	394 }
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	395
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	396 /* start soft connect time-out (but only if we have another IP left) */
19039 8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	397 if (queue->addr.type != HTTP_CLIENT_PEER_ADDR_UNIX) {
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	398 msecs = host->client->set.soft_connect_timeout_msecs;
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	399 if (!http_client_queue_is_last_connect_ip(queue) && msecs > 0 &&
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	400 queue->to_connect == NULL) {
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	401 queue->to_connect =
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	402 timeout_add(msecs, http_client_queue_soft_connect_timeout, queue);
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	403 }
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	404 }
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	405 }
21044 66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	406
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	407 return peer;
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	408 }
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	409
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	410 void http_client_queue_connection_setup(struct http_client_queue *queue)
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	411 {
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	412 (void)http_client_queue_connection_attempt(queue);
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	413 }
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	414
21047 3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	415 unsigned int
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	416 http_client_queue_host_lookup_done(struct http_client_queue *queue)
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	417 {
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	418 unsigned int reqs_pending =
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	419 http_client_queue_requests_pending(queue, NULL);
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	420 http_client_queue_recover_from_lookup(queue);
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	421 if (reqs_pending > 0)
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	422 http_client_queue_connection_setup(queue);
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	423 return reqs_pending;
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	424 }
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	425
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	426 void
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	427 http_client_queue_connection_success(struct http_client_queue *queue,
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	428 const struct http_client_peer_addr *addr)
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	429 {
21047 3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	430 if (queue->host->dns_lookup == NULL &&
3a481a8f2ff2 lib-http: client: Implemented host name lookup TTL. Stephan Bosch <stephan@dovecot.fi> parents: 21046 diff changeset	431 queue->addr.type != HTTP_CLIENT_PEER_ADDR_UNIX) {
19039 8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	432 /* we achieved at least one connection the the addr->ip */
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	433 queue->ips_connect_start_idx =
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	434 http_client_host_get_ip_idx(queue->host, &addr->a.tcp.ip);
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	435 }
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	436
17774 650629de6a73 lib-http: client: Added support for attempting a single IP several times. Stephan Bosch <stephan@rename-it.nl> parents: 17172 diff changeset	437 /* reset attempt counter */
650629de6a73 lib-http: client: Added support for attempting a single IP several times. Stephan Bosch <stephan@rename-it.nl> parents: 17172 diff changeset	438 queue->connect_attempts = 0;
650629de6a73 lib-http: client: Added support for attempting a single IP several times. Stephan Bosch <stephan@rename-it.nl> parents: 17172 diff changeset	439
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	440 /* stop soft connect time-out */
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	441 if (queue->to_connect != NULL)
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	442 timeout_remove(&queue->to_connect);
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	443
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	444 /* drop all other attempts to the hport. note that we get here whenever
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	445 a connection is successfully created, so pending_peers array
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	446 may be empty. */
21045 c926a09729f0 lib-http: client: Create queue->pending_peers array from the beginning. Stephan Bosch <stephan@dovecot.fi> parents: 21044 diff changeset	447 if (array_count(&queue->pending_peers) > 0) {
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	448 struct http_client_peer const peer_idx;
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	449
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	450 array_foreach(&queue->pending_peers, peer_idx) {
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	451 if (http_client_peer_addr_cmp(&(*peer_idx)->addr, addr) == 0) {
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	452 /* don't drop any connections to the successfully
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	453 connected peer, even if some of the connections
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	454 are pending. they may be intended for urgent
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	455 requests. */
21044 66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	456 i_assert(queue->cur_peer == NULL);
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	457 queue->cur_peer = *peer_idx;
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	458 continue;
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	459 }
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	460 /* unlink this queue from the peer; if this was the last/only queue, the
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	461 peer will be freed, closing all connections.
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	462 */
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	463 http_client_peer_unlink_queue(*peer_idx, queue);
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	464 }
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	465 array_clear(&queue->pending_peers);
21044 66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	466 i_assert(queue->cur_peer != NULL);
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	467 }
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	468 }
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	469
17969 c9ea647cba87 lib-http: client: Fixed behavior for max_connect_attempts with fewer IPs than attempts. Stephan Bosch <stephan@rename-it.nl> parents: 17942 diff changeset	470 void
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	471 http_client_queue_connection_failure(struct http_client_queue *queue,
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	472 const struct http_client_peer_addr addr, const char reason)
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	473 {
17969 c9ea647cba87 lib-http: client: Fixed behavior for max_connect_attempts with fewer IPs than attempts. Stephan Bosch <stephan@rename-it.nl> parents: 17942 diff changeset	474 const struct http_client_settings *set =
c9ea647cba87 lib-http: client: Fixed behavior for max_connect_attempts with fewer IPs than attempts. Stephan Bosch <stephan@rename-it.nl> parents: 17942 diff changeset	475 &queue->client->set;
19039 8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	476 const char *https_name = http_client_peer_addr_get_https_name(addr);
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	477 struct http_client_host *host = queue->host;
21044 66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	478 struct http_client_peer *failed_peer;
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	479 struct http_client_peer const peer_idx;
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	480
19039 8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	481 http_client_queue_debug(queue,
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	482 "Failed to set up connection to %s%s: %s "
17171 22424f1a32ad lib-http: Added debug logging. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	483 "(%u peers pending, %u requests pending)",
22424f1a32ad lib-http: Added debug logging. Timo Sirainen <tss@iki.fi> parents: 17130 diff changeset	484 http_client_peer_addr2str(addr),
19039 8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	485 (https_name == NULL ? "" :
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	486 t_strdup_printf(" (SSL=%s)", https_name)),
21045 c926a09729f0 lib-http: client: Create queue->pending_peers array from the beginning. Stephan Bosch <stephan@dovecot.fi> parents: 21044 diff changeset	487 reason, array_count(&queue->pending_peers),
17881 4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	488 array_count(&queue->requests));
17969 c9ea647cba87 lib-http: client: Fixed behavior for max_connect_attempts with fewer IPs than attempts. Stephan Bosch <stephan@rename-it.nl> parents: 17942 diff changeset	489
21466 10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	490 if (queue->cur_peer != NULL) {
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	491 if (http_client_peer_is_connected(queue->cur_peer)) {
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	492 /* The peer still has some working connections, which means that
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	493 pending requests wait until they're picked up by those connections
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	494 or the remaining connections fail as well. In the latter case,
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	495 connecting to different peer can resolve the situation, but only
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	496 if there is more than one IP. In any other case, the requests will
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	497 eventually fail. In the future we could start connections to the next
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	498 IP at this point already, but that is no small change. */
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	499 i_assert(array_count(&queue->pending_peers) == 0);
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	500 return;
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	501 }
21466 10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	502
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	503 failed_peer = queue->cur_peer;
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	504 http_client_peer_unlink_queue(queue->cur_peer, queue);
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	505 queue->cur_peer = NULL;
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	506
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	507 } else {
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	508 /* we're still doing the initial connections to this hport. if
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	509 we're also doing parallel connections with soft timeouts
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	510 (pending_peer_count>1), wait for them to finish
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	511 first. */
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	512 failed_peer = NULL;
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	513 array_foreach(&queue->pending_peers, peer_idx) {
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	514 if (http_client_peer_addr_cmp(&(*peer_idx)->addr, addr) == 0) {
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	515 failed_peer = *peer_idx;
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	516 array_delete(&queue->pending_peers,
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	517 array_foreach_idx(&queue->pending_peers, peer_idx), 1);
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	518 break;
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	519 }
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	520 }
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	521 i_assert(failed_peer != NULL);
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	522 if (array_count(&queue->pending_peers) > 0) {
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	523 http_client_queue_debug(queue,
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	524 "Waiting for remaining pending peers.");
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	525 http_client_peer_unlink_queue(failed_peer, queue);
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	526 return;
10103d3a4ad4 lib-http: client: Fixed peer reconnection failure handling. Stephan Bosch <stephan.bosch@dovecot.fi> parents: 21390 diff changeset	527 }
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	528 }
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	529
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	530 /* one of the connections failed. if we're not using soft timeouts,
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	531 we need to try to connect to the next IP. if we are using soft
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	532 timeouts, we've already tried all of the IPs by now. */
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	533 if (queue->to_connect != NULL)
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	534 timeout_remove(&queue->to_connect);
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	535
19039 8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	536 if (queue->addr.type == HTTP_CLIENT_PEER_ADDR_UNIX) {
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	537 http_client_queue_fail(queue,
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	538 HTTP_CLIENT_REQUEST_ERROR_CONNECT_FAILED, reason);
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	539 return;
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	540 }
8f8f768937f5 http-client: Added support for using an HTTP proxy running on a unix socket. Stephan Bosch <stephan@rename-it.nl> parents: 18137 diff changeset	541
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	542 if (http_client_queue_is_last_connect_ip(queue)) {
17969 c9ea647cba87 lib-http: client: Fixed behavior for max_connect_attempts with fewer IPs than attempts. Stephan Bosch <stephan@rename-it.nl> parents: 17942 diff changeset	543 /* all IPs failed, but retry all of them again if we have more
c9ea647cba87 lib-http: client: Fixed behavior for max_connect_attempts with fewer IPs than attempts. Stephan Bosch <stephan@rename-it.nl> parents: 17942 diff changeset	544 connect attempts left or on the next request. */
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	545 queue->ips_connect_idx = queue->ips_connect_start_idx =
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	546 (queue->ips_connect_idx + 1) % host->ips_count;
17969 c9ea647cba87 lib-http: client: Fixed behavior for max_connect_attempts with fewer IPs than attempts. Stephan Bosch <stephan@rename-it.nl> parents: 17942 diff changeset	547
c9ea647cba87 lib-http: client: Fixed behavior for max_connect_attempts with fewer IPs than attempts. Stephan Bosch <stephan@rename-it.nl> parents: 17942 diff changeset	548 if (set->max_connect_attempts == 0 \|\|
c9ea647cba87 lib-http: client: Fixed behavior for max_connect_attempts with fewer IPs than attempts. Stephan Bosch <stephan@rename-it.nl> parents: 17942 diff changeset	549 queue->connect_attempts >= set->max_connect_attempts) {
c9ea647cba87 lib-http: client: Fixed behavior for max_connect_attempts with fewer IPs than attempts. Stephan Bosch <stephan@rename-it.nl> parents: 17942 diff changeset	550 http_client_queue_debug(queue,
c9ea647cba87 lib-http: client: Fixed behavior for max_connect_attempts with fewer IPs than attempts. Stephan Bosch <stephan@rename-it.nl> parents: 17942 diff changeset	551 "Failed to set up any connection; failing all queued requests");
20152 b983a3ab3fcf lib-http: If connect fails, include attempt count and total time in error. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 19552 diff changeset	552 if (queue->connect_attempts > 1) {
b983a3ab3fcf lib-http: If connect fails, include attempt count and total time in error. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 19552 diff changeset	553 unsigned int total_msecs =
b983a3ab3fcf lib-http: If connect fails, include attempt count and total time in error. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 19552 diff changeset	554 timeval_diff_msecs(&ioloop_timeval, &queue->first_connect_time);
b983a3ab3fcf lib-http: If connect fails, include attempt count and total time in error. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 19552 diff changeset	555 reason = t_strdup_printf("%s (%u attempts in %u.%03u secs)",
b983a3ab3fcf lib-http: If connect fails, include attempt count and total time in error. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 19552 diff changeset	556 reason, queue->connect_attempts,
b983a3ab3fcf lib-http: If connect fails, include attempt count and total time in error. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 19552 diff changeset	557 total_msecs/1000, total_msecs%1000);
b983a3ab3fcf lib-http: If connect fails, include attempt count and total time in error. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 19552 diff changeset	558 }
17969 c9ea647cba87 lib-http: client: Fixed behavior for max_connect_attempts with fewer IPs than attempts. Stephan Bosch <stephan@rename-it.nl> parents: 17942 diff changeset	559 queue->connect_attempts = 0;
21044 66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	560 http_client_peer_unlink_queue(failed_peer, queue);
17969 c9ea647cba87 lib-http: client: Fixed behavior for max_connect_attempts with fewer IPs than attempts. Stephan Bosch <stephan@rename-it.nl> parents: 17942 diff changeset	561 http_client_queue_fail(queue,
c9ea647cba87 lib-http: client: Fixed behavior for max_connect_attempts with fewer IPs than attempts. Stephan Bosch <stephan@rename-it.nl> parents: 17942 diff changeset	562 HTTP_CLIENT_REQUEST_ERROR_CONNECT_FAILED, reason);
c9ea647cba87 lib-http: client: Fixed behavior for max_connect_attempts with fewer IPs than attempts. Stephan Bosch <stephan@rename-it.nl> parents: 17942 diff changeset	563 return;
c9ea647cba87 lib-http: client: Fixed behavior for max_connect_attempts with fewer IPs than attempts. Stephan Bosch <stephan@rename-it.nl> parents: 17942 diff changeset	564 }
c9ea647cba87 lib-http: client: Fixed behavior for max_connect_attempts with fewer IPs than attempts. Stephan Bosch <stephan@rename-it.nl> parents: 17942 diff changeset	565 } else {
c9ea647cba87 lib-http: client: Fixed behavior for max_connect_attempts with fewer IPs than attempts. Stephan Bosch <stephan@rename-it.nl> parents: 17942 diff changeset	566 queue->ips_connect_idx = (queue->ips_connect_idx + 1) % host->ips_count;
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	567 }
17969 c9ea647cba87 lib-http: client: Fixed behavior for max_connect_attempts with fewer IPs than attempts. Stephan Bosch <stephan@rename-it.nl> parents: 17942 diff changeset	568
21044 66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	569 if (http_client_queue_connection_attempt(queue) != failed_peer)
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	570 http_client_peer_unlink_queue(failed_peer, queue);
17969 c9ea647cba87 lib-http: client: Fixed behavior for max_connect_attempts with fewer IPs than attempts. Stephan Bosch <stephan@rename-it.nl> parents: 17942 diff changeset	571 return;
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	572 }
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	573
21035 1590d1ff0551 lib-http: client: Unlink all queues from peer when it is disconnected. Stephan Bosch <stephan@dovecot.fi> parents: 20344 diff changeset	574 void
1590d1ff0551 lib-http: client: Unlink all queues from peer when it is disconnected. Stephan Bosch <stephan@dovecot.fi> parents: 20344 diff changeset	575 http_client_queue_peer_disconnected(struct http_client_queue *queue,
1590d1ff0551 lib-http: client: Unlink all queues from peer when it is disconnected. Stephan Bosch <stephan@dovecot.fi> parents: 20344 diff changeset	576 struct http_client_peer *peer)
1590d1ff0551 lib-http: client: Unlink all queues from peer when it is disconnected. Stephan Bosch <stephan@dovecot.fi> parents: 20344 diff changeset	577 {
1590d1ff0551 lib-http: client: Unlink all queues from peer when it is disconnected. Stephan Bosch <stephan@dovecot.fi> parents: 20344 diff changeset	578 struct http_client_peer const peer_idx;
1590d1ff0551 lib-http: client: Unlink all queues from peer when it is disconnected. Stephan Bosch <stephan@dovecot.fi> parents: 20344 diff changeset	579
21044 66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	580 if (queue->cur_peer == peer) {
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	581 queue->cur_peer = NULL;
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	582 return;
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	583 }
66c79e8a1132 lib-http: client: Link peers to queue earlier: during connection attempts. Stephan Bosch <stephan@dovecot.fi> parents: 21035 diff changeset	584
21035 1590d1ff0551 lib-http: client: Unlink all queues from peer when it is disconnected. Stephan Bosch <stephan@dovecot.fi> parents: 20344 diff changeset	585 array_foreach(&queue->pending_peers, peer_idx) {
1590d1ff0551 lib-http: client: Unlink all queues from peer when it is disconnected. Stephan Bosch <stephan@dovecot.fi> parents: 20344 diff changeset	586 if (*peer_idx == peer) {
1590d1ff0551 lib-http: client: Unlink all queues from peer when it is disconnected. Stephan Bosch <stephan@dovecot.fi> parents: 20344 diff changeset	587 array_delete(&queue->pending_peers,
1590d1ff0551 lib-http: client: Unlink all queues from peer when it is disconnected. Stephan Bosch <stephan@dovecot.fi> parents: 20344 diff changeset	588 array_foreach_idx(&queue->pending_peers, peer_idx), 1);
1590d1ff0551 lib-http: client: Unlink all queues from peer when it is disconnected. Stephan Bosch <stephan@dovecot.fi> parents: 20344 diff changeset	589 break;
1590d1ff0551 lib-http: client: Unlink all queues from peer when it is disconnected. Stephan Bosch <stephan@dovecot.fi> parents: 20344 diff changeset	590 }
1590d1ff0551 lib-http: client: Unlink all queues from peer when it is disconnected. Stephan Bosch <stephan@dovecot.fi> parents: 20344 diff changeset	591 }
1590d1ff0551 lib-http: client: Unlink all queues from peer when it is disconnected. Stephan Bosch <stephan@dovecot.fi> parents: 20344 diff changeset	592 }
1590d1ff0551 lib-http: client: Unlink all queues from peer when it is disconnected. Stephan Bosch <stephan@dovecot.fi> parents: 20344 diff changeset	593
17881 4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	594 /*
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	595 * Main request queue
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	596 */
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	597
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	598 void
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	599 http_client_queue_drop_request(struct http_client_queue *queue,
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	600 struct http_client_request *req)
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	601 {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	602 struct http_client_request **reqs;
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	603 unsigned int count, i;
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	604
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	605 http_client_queue_debug(queue,
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	606 "Dropping request %s", http_client_request_label(req));
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	607
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	608 /* drop from queue */
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	609 if (req->urgent) {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	610 reqs = array_get_modifiable(&queue->queued_urgent_requests, &count);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	611 for (i = 0; i < count; i++) {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	612 if (reqs[i] == req) {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	613 array_delete(&queue->queued_urgent_requests, i, 1);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	614 break;
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	615 }
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	616 }
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	617 } else {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	618 reqs = array_get_modifiable(&queue->queued_requests, &count);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	619 for (i = 0; i < count; i++) {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	620 if (reqs[i] == req) {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	621 array_delete(&queue->queued_requests, i, 1);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	622 break;
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	623 }
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	624 }
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	625 }
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	626
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	627 /* drop from delay queue */
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	628 if (req->release_time.tv_sec > 0) {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	629 reqs = array_get_modifiable(&queue->delayed_requests, &count);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	630 for (i = 0; i < count; i++) {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	631 if (reqs[i] == req)
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	632 break;
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	633 }
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	634 if (i < count) {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	635 if (i == 0) {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	636 if (queue->to_delayed != NULL) {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	637 timeout_remove(&queue->to_delayed);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	638 if (count > 1) {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	639 i_assert(reqs[1]->release_time.tv_sec > 0);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	640 http_client_queue_set_request_timer(queue, &reqs[1]->release_time);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	641 }
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	642 }
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	643 }
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	644 array_delete(&queue->delayed_requests, i, 1);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	645 }
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	646 }
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	647
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	648 /* drop from main request list */
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	649 reqs = array_get_modifiable(&queue->requests, &count);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	650 for (i = 0; i < count; i++) {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	651 if (reqs[i] == req)
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	652 break;
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	653 }
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	654 i_assert(i < count);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	655
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	656 if (i == 0) {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	657 if (queue->to_request != NULL) {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	658 timeout_remove(&queue->to_request);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	659 if (count > 1 && reqs[1]->timeout_time.tv_sec > 0)
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	660 http_client_queue_set_request_timer(queue, &reqs[1]->timeout_time);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	661 }
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	662 }
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	663 req->queue = NULL;
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	664 array_delete(&queue->requests, i, 1);
21048 4180c1dd0aa2 lib-http: client: Destroy host and associated queue objects after no more requests remain and an idle timeout expires. Stephan Bosch <stephan@dovecot.fi> parents: 21047 diff changeset	665
4180c1dd0aa2 lib-http: client: Destroy host and associated queue objects after no more requests remain and an idle timeout expires. Stephan Bosch <stephan@dovecot.fi> parents: 21047 diff changeset	666 if (array_count(&queue->requests) == 0)
4180c1dd0aa2 lib-http: client: Destroy host and associated queue objects after no more requests remain and an idle timeout expires. Stephan Bosch <stephan@dovecot.fi> parents: 21047 diff changeset	667 http_client_host_check_idle(queue->host);
17881 4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	668 return;
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	669 }
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	670
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	671 static void
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	672 http_client_queue_request_timeout(struct http_client_queue *queue)
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	673 {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	674 struct http_client_request const reqs;
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	675 ARRAY_TYPE(http_client_request) failed_requests;
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	676 struct timeval new_to = { 0, 0 };
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	677 unsigned int count, i;
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	678
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	679 http_client_queue_debug(queue, "Timeout (now: %s.%03lu)",
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	680 t_strflocaltime("%Y-%m-%d %H:%M:%S", ioloop_timeval.tv_sec),
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	681 ((unsigned long)ioloop_timeval.tv_usec)/1000);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	682
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	683 if (queue->to_request != NULL)
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	684 timeout_remove(&queue->to_request);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	685
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	686 /* collect failed requests */
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	687 reqs = array_get(&queue->requests, &count);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	688 i_assert(count > 0);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	689 t_array_init(&failed_requests, count);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	690 for (i = 0; i < count; i++) {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	691 if (reqs[i]->timeout_time.tv_sec > 0 &&
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	692 timeval_cmp_margin(&reqs[i]->timeout_time,
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	693 &ioloop_timeval, TIMEOUT_CMP_MARGIN_USECS) > 0) {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	694 break;
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	695 }
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	696 array_append(&failed_requests, &reqs[i], 1);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	697 }
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	698
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	699 /* update timout */
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	700 if (i < count)
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	701 new_to = reqs[i]->timeout_time;
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	702
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	703 /* abort all failed request */
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	704 reqs = array_get(&failed_requests, &count);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	705 i_assert(count > 0); /* at least one request timed out */
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	706 for (i = 0; i < count; i++) {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	707 struct http_client_request *req = reqs[i];
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	708
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	709 http_client_queue_debug(queue,
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	710 "Request %s timed out", http_client_request_label(req));
20344 2b71a1c2febc lib-http: Changed http_client_request_error to set request to NULL Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20332 diff changeset	711 http_client_request_error(&req,
17881 4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	712 HTTP_CLIENT_REQUEST_ERROR_TIMED_OUT,
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	713 "Timed out");
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	714 }
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	715
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	716 if (new_to.tv_sec > 0) {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	717 http_client_queue_debug(queue, "New timeout");
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	718 http_client_queue_set_request_timer(queue, &new_to);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	719 }
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	720 }
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	721
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	722 static void
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	723 http_client_queue_set_request_timer(struct http_client_queue *queue,
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	724 const struct timeval *time)
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	725 {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	726 i_assert(time->tv_sec > 0);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	727 if (queue->to_request != NULL)
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	728 timeout_remove(&queue->to_request);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	729
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	730 if (queue->client->set.debug) {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	731 http_client_queue_debug(queue,
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	732 "Set request timeout to %s.%03lu (now: %s.%03lu)",
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	733 t_strflocaltime("%Y-%m-%d %H:%M:%S", time->tv_sec),
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	734 ((unsigned long)time->tv_usec)/1000,
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	735 t_strflocaltime("%Y-%m-%d %H:%M:%S", ioloop_timeval.tv_sec),
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	736 ((unsigned long)ioloop_timeval.tv_usec)/1000);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	737 }
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	738
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	739 /* set timer */
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	740 queue->to_request = timeout_add_absolute
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	741 (time, http_client_queue_request_timeout, queue);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	742 }
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	743
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	744 static int
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	745 http_client_queue_request_timeout_cmp(struct http_client_request const req1,
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	746 struct http_client_request const req2)
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	747 {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	748 int ret;
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	749
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	750 /* 0 means no timeout */
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	751 if ((*req1)->timeout_time.tv_sec == 0) {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	752 if ((*req2)->timeout_time.tv_sec == 0) {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	753 /* sort by age */
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	754 if ((ret=timeval_cmp(&(req1)->submit_time, &(req2)->submit_time)) != 0)
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	755 return ret;
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	756
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	757 } else {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	758 return 1;
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	759 }
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	760 } else if ((*req2)->timeout_time.tv_sec == 0) {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	761 return -1;
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	762
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	763 /* sort by timeout */
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	764 } else if
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	765 ((ret=timeval_cmp(&(req1)->timeout_time, &(req2)->timeout_time)) != 0) {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	766 return ret;
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	767 }
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	768
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	769 /* sort by minumum attempts for fairness */
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	770 return ((req2)->attempts - (req1)->attempts);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	771 }
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	772
17005 d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	773 static void http_client_queue_submit_now(struct http_client_queue *queue,
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	774 struct http_client_request *req)
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	775 {
17881 4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	776 ARRAY_TYPE(http_client_request) *req_queue;
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	777
17005 d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	778 req->release_time.tv_sec = 0;
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	779 req->release_time.tv_usec = 0;
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	780
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	781 if (req->urgent)
17881 4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	782 req_queue = &queue->queued_urgent_requests;
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	783 else
17881 4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	784 req_queue = &queue->queued_requests;
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	785
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	786 /* enqueue */
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	787 if (req->timeout_time.tv_sec == 0) {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	788 /* no timeout; enqueue at end */
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	789 array_append(req_queue, &req, 1);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	790
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	791 } else if (timeval_diff_msecs(&req->timeout_time, &ioloop_timeval) <= 1) {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	792 /* pretty much already timed out; don't bother */
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	793
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	794 } else {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	795 unsigned int insert_idx;
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	796
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	797 /* keep transmission queue sorted earliest timeout first */
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	798 (void)array_bsearch_insert_pos(req_queue,
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	799 &req, http_client_queue_request_timeout_cmp, &insert_idx);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	800 array_insert(req_queue, insert_idx, &req, 1);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	801 }
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	802 }
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	803
17881 4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	804 /*
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	805 * Delayed request queue
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	806 */
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	807
17005 d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	808 static void
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	809 http_client_queue_delay_timeout(struct http_client_queue *queue)
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	810 {
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	811 struct http_client_request const reqs;
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	812 unsigned int count, i, finished;
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	813
20166 b65cd55896a7 lib-http: Queue's delayed timeout handler wasn't removed after work was done. Timo Sirainen <timo.sirainen@dovecot.fi> parents: 20152 diff changeset	814 timeout_remove(&queue->to_delayed);
17005 d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	815 io_loop_time_refresh();
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	816
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	817 finished = 0;
17881 4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	818 reqs = array_get(&queue->delayed_requests, &count);
17005 d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	819 for (i = 0; i < count; i++) {
17879 94ba0150905b lib-http: client queue: Start using new timeval_cmp_margin function for delay handling. Stephan Bosch <stephan@rename-it.nl> parents: 17852 diff changeset	820 if (timeval_cmp_margin(&reqs[i]->release_time,
94ba0150905b lib-http: client queue: Start using new timeval_cmp_margin function for delay handling. Stephan Bosch <stephan@rename-it.nl> parents: 17852 diff changeset	821 &ioloop_timeval, TIMEOUT_CMP_MARGIN_USECS) > 0) {
17005 d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	822 break;
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	823 }
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	824
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	825 http_client_queue_debug(queue,
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	826 "Activated delayed request %s%s",
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	827 http_client_request_label(reqs[i]),
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	828 (reqs[i]->urgent ? " (urgent)" : ""));
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	829 http_client_queue_submit_now(queue, reqs[i]);
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	830 finished++;
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	831 }
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	832 if (i < count) {
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	833 http_client_queue_set_delay_timer(queue, reqs[i]->release_time);
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	834 }
17881 4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	835 array_delete(&queue->delayed_requests, 0, finished);
17005 d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	836
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	837 http_client_queue_connection_setup(queue);
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	838 }
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	839
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	840 static void
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	841 http_client_queue_set_delay_timer(struct http_client_queue *queue,
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	842 struct timeval time)
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	843 {
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	844 int usecs = timeval_diff_usecs(&time, &ioloop_timeval);
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	845 int msecs;
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	846
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	847 /* round up to nearest microsecond */
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	848 msecs = (usecs + 999) / 1000;
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	849
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	850 /* set timer */
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	851 if (queue->to_delayed != NULL)
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	852 timeout_remove(&queue->to_delayed);
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	853 queue->to_delayed = timeout_add
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	854 (msecs, http_client_queue_delay_timeout, queue);
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	855 }
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	856
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	857 static int
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	858 http_client_queue_delayed_cmp(struct http_client_request const req1,
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	859 struct http_client_request const req2)
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	860 {
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	861 return timeval_cmp(&(req1)->release_time, &(req2)->release_time);
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	862 }
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	863
17881 4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	864 /*
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	865 * Request submission
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	866 */
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	867
17005 d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	868 void http_client_queue_submit_request(struct http_client_queue *queue,
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	869 struct http_client_request *req)
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	870 {
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	871 unsigned int insert_idx;
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	872
17881 4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	873 if (req->queue != NULL)
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	874 http_client_queue_drop_request(req->queue, req);
17005 d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	875 req->queue = queue;
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	876
17881 4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	877 /* check delay vs timeout */
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	878 if (req->release_time.tv_sec > 0 && req->timeout_time.tv_sec > 0 &&
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	879 timeval_cmp_margin(&req->release_time,
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	880 &req->timeout_time, TIMEOUT_CMP_MARGIN_USECS) >= 0) {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	881 /* release time is later than absolute timeout */
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	882 req->release_time.tv_sec = 0;
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	883 req->release_time.tv_usec = 0;
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	884
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	885 /* timeout rightaway */
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	886 req->timeout_time = ioloop_timeval;
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	887
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	888 http_client_queue_debug(queue,
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	889 "Delayed request %s%s already timed out",
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	890 http_client_request_label(req),
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	891 (req->urgent ? " (urgent)" : ""));
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	892 }
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	893
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	894 /* add to main request list */
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	895 if (req->timeout_time.tv_sec == 0) {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	896 /* no timeout; just append */
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	897 array_append(&queue->requests, &req, 1);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	898
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	899 } else {
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	900 unsigned int insert_idx;
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	901
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	902 /* keep main request list sorted earliest timeout first */
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	903 (void)array_bsearch_insert_pos(&queue->requests,
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	904 &req, http_client_queue_request_timeout_cmp, &insert_idx);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	905 array_insert(&queue->requests, insert_idx, &req, 1);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	906
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	907 /* now first in queue; update timer */
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	908 if (insert_idx == 0)
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	909 http_client_queue_set_request_timer(queue, &req->timeout_time);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	910 }
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	911
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	912 /* handle delay */
17005 d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	913 if (req->release_time.tv_sec > 0) {
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	914 io_loop_time_refresh();
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	915
17879 94ba0150905b lib-http: client queue: Start using new timeval_cmp_margin function for delay handling. Stephan Bosch <stephan@rename-it.nl> parents: 17852 diff changeset	916 if (timeval_cmp_margin(&req->release_time,
94ba0150905b lib-http: client queue: Start using new timeval_cmp_margin function for delay handling. Stephan Bosch <stephan@rename-it.nl> parents: 17852 diff changeset	917 &ioloop_timeval, TIMEOUT_CMP_MARGIN_USECS) > 0) {
20332 498fec031039 lib-http: client: Added more debug logging about the submission of delayed requests. Stephan Bosch <stephan@dovecot.fi> parents: 20166 diff changeset	918 http_client_queue_debug(queue,
498fec031039 lib-http: client: Added more debug logging about the submission of delayed requests. Stephan Bosch <stephan@dovecot.fi> parents: 20166 diff changeset	919 "Delayed request %s%s submitted (time remaining: %d msecs)",
498fec031039 lib-http: client: Added more debug logging about the submission of delayed requests. Stephan Bosch <stephan@dovecot.fi> parents: 20166 diff changeset	920 http_client_request_label(req),
498fec031039 lib-http: client: Added more debug logging about the submission of delayed requests. Stephan Bosch <stephan@dovecot.fi> parents: 20166 diff changeset	921 (req->urgent ? " (urgent)" : ""),
498fec031039 lib-http: client: Added more debug logging about the submission of delayed requests. Stephan Bosch <stephan@dovecot.fi> parents: 20166 diff changeset	922 timeval_diff_msecs(&req->release_time, &ioloop_timeval));
498fec031039 lib-http: client: Added more debug logging about the submission of delayed requests. Stephan Bosch <stephan@dovecot.fi> parents: 20166 diff changeset	923
17881 4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	924 (void)array_bsearch_insert_pos(&queue->delayed_requests,
17005 d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	925 &req, http_client_queue_delayed_cmp, &insert_idx);
17881 4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	926 array_insert(&queue->delayed_requests, insert_idx, &req, 1);
17005 d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	927 if (insert_idx == 0)
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	928 http_client_queue_set_delay_timer(queue, req->release_time);
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	929 return;
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	930 }
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	931 }
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	932
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	933 http_client_queue_submit_now(queue, req);
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	934 }
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	935
17881 4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	936 /*
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	937 * Request retrieval
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	938 */
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	939
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	940 struct http_client_request *
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	941 http_client_queue_claim_request(struct http_client_queue *queue,
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	942 const struct http_client_peer_addr *addr, bool no_urgent)
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	943 {
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	944 struct http_client_request const requests;
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	945 struct http_client_request *req;
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	946 unsigned int i, count;
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	947
17881 4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	948 count = 0;
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	949 if (!no_urgent)
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	950 requests = array_get(&queue->queued_urgent_requests, &count);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	951
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	952 if (count == 0)
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	953 requests = array_get(&queue->queued_requests, &count);
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	954 if (count == 0)
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	955 return NULL;
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	956 i = 0;
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	957 req = requests[i];
17881 4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	958 if (req->urgent)
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	959 array_delete(&queue->queued_urgent_requests, i, 1);
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	960 else
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	961 array_delete(&queue->queued_requests, i, 1);
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	962
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	963 http_client_queue_debug(queue,
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	964 "Connection to peer %s claimed request %s %s",
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	965 http_client_peer_addr2str(addr), http_client_request_label(req),
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	966 (req->urgent ? "(urgent)" : ""));
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	967
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	968 return req;
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	969 }
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	970
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	971 unsigned int
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	972 http_client_queue_requests_pending(struct http_client_queue *queue,
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	973 unsigned int *num_urgent_r)
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	974 {
17881 4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	975 unsigned int urg_count = array_count(&queue->queued_urgent_requests);
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	976
17881 4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	977 if (num_urgent_r != NULL)
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	978 *num_urgent_r = urg_count;
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	979 return array_count(&queue->queued_requests) + urg_count;
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	980 }
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	981
21046 5b3dc5fef64c lib-http: client: Added http_client_queue_requests_active(). Stephan Bosch <stephan@dovecot.fi> parents: 21045 diff changeset	982 unsigned int
5b3dc5fef64c lib-http: client: Added http_client_queue_requests_active(). Stephan Bosch <stephan@dovecot.fi> parents: 21045 diff changeset	983 http_client_queue_requests_active(struct http_client_queue *queue)
5b3dc5fef64c lib-http: client: Added http_client_queue_requests_active(). Stephan Bosch <stephan@dovecot.fi> parents: 21045 diff changeset	984 {
5b3dc5fef64c lib-http: client: Added http_client_queue_requests_active(). Stephan Bosch <stephan@dovecot.fi> parents: 21045 diff changeset	985 return array_count(&queue->requests);
5b3dc5fef64c lib-http: client: Added http_client_queue_requests_active(). Stephan Bosch <stephan@dovecot.fi> parents: 21045 diff changeset	986 }
5b3dc5fef64c lib-http: client: Added http_client_queue_requests_active(). Stephan Bosch <stephan@dovecot.fi> parents: 21045 diff changeset	987
17881 4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	988 /*
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	989 * ioloop
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	990 */
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	991
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	992 void http_client_queue_switch_ioloop(struct http_client_queue *queue)
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	993 {
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	994 if (queue->to_connect != NULL)
160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	995 queue->to_connect = io_loop_move_timeout(&queue->to_connect);
17881 4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	996 if (queue->to_request != NULL)
4f175c27bea5 lib-http: client: Added support for absolute request timeout. Stephan Bosch <stephan@rename-it.nl> parents: 17879 diff changeset	997 queue->to_request = io_loop_move_timeout(&queue->to_request);
17005 d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	998 if (queue->to_delayed != NULL)
d2f2423615d7 http: Implemented delayed requests scheduling. Stephan Bosch <stephan@rename-it.nl> parents: 17002 diff changeset	999 queue->to_delayed = io_loop_move_timeout(&queue->to_delayed);
17002 160e489d7c12 http-client: Changed struct http_client_host_port into a struct http_client_queue object. Stephan Bosch <stephan@rename-it.nl> parents: diff changeset	1000 }

Mercurial > dovecot > core-2.2

annotate src/lib-http/http-client-queue.c @ 21466:10103d3a4ad4