ϪfdZddlmZgdZddlZddlZddlZddlZddlZddl Z ddl Z ddl Z ddl Z ddl mZddlmZddlmZddlmZmZmZmZmZmZdd lmZmZmZdd l m!Z!m"Z"m#Z#m$Z$dd l%m&Z&dd l'm(Z(m)Z)m*Z*dd l+m,Z,ddl-m.Z.ddl/m0Z0ddl1m2Z2ddl3m4Z4m5Z5ddl6m7Z7ddl8m9Z9m:Z:ddl;mZ>ddl?m@Z@ddlAmBZBmCZCmDZDmEZEmFZFmGZGmHZHmIZImJZJmKZKmLZLmMZMmNZNmOZOmPZPmQZQmRZRmSZSmTZTmUZUmVZVmWZWmXZXmYZYmZZZm[Z[m\Z\m]Z]m^Z^m_Z_m`Z`maZambZbmcZcmdZdmeZemfZfmgZgmhZhmiZimjZjmkZkmlZlmmZmddlnmoZompZpddlqmrZrmsZsmtZt ddlumvZvdZwdZydZzdZ{efZ|gd Z}gd!Z~e}Dcgc]}|jc}Ze~Dcgc]}|xr|jc}ZdZd"ZGd#d$eZd[d%Zd&Zd\d'Zd]d(Zd]d)Zd*Zd+Zd,Zd^d-Zd_d.Zd`d/Zd0ZGd1d2e"ZGd3d4ZGd5d6e4j$Zd7ZeZd8Ze j0d9Ze#e)j4eGd:d;ZGd<d=eZGd>d?eZGd@dAeZGdBdCZdDZdEZ GdFdGZe#e)jFGdHdIZe#e)jJe)jFe)j4GdJdKe4j$e5jLZdLZe$erdMZe#e)jTGdNdOZGdPdQe BRR BR BE B  ::dA&DAq ! $-.A $-.AAv! As!  H C   sB2B88 CCc |dk(rtj}tj|\ }}}}}}}}} tdt||t|||||fz} | S)zT Convert seconds since epoch to HTTP datetime string. @rtype: C{bytes} Nz#%s, %02d %3s %4d %02d:%02d:%02d GMT)timegmtimer\ weekdayname monthname msSinceEpochyearmonthdayhhmmsswdyzss rr3r3?sm tyy{ -1[[-F*D%b"b"a- r?C5!14R D E A Hrc |dk(rtj}tj|\ }}}}}}}}} d|t|||||fz} | S)zQ Convert seconds since epoch to log datetime string. @rtype: C{str} Nz#[%02d/%3s/%4d:%02d:%02d:%02d +0000])rrrrs rr4r4Ose tyy{ -1[[-F*D%b"b"a- %    1 A Hrc\d}||krtd|fzd|cxkrdksJJd||z ztj||z}td|D]}|tj|z}|dkDrtj |r|dz}||zdz }|dz|z} | dz|z} | dz|z} | S) z? Convert time tuple in GMT to seconds since epoch, GMT izYears prior to %d not supportedr imrf)rcalendarleapdaysrangemdaysisleap) rrrhourminutesecondEPOCHdaysihoursminutessecondss rr5r5cs E e|:eXEFF      $, ("3"3E4"@ @D 1e_(hnnQ''( qyX__T*ax #:>D 2I Ebj6!GlV#G Nrc Bt|j}|dddjtvr t d|zSt |}|dk(s|dk(r(|djr|d}|d}|d}|d}n|dk(s|dk(rU|djd d k7r>|djd \}}}|d}t|}|d kr|d z}n<|d kr7|dz}n1t |dk(r|d}|d}|d}|d}nt d|zt|}ttj|j}t|}tt|jd\}}} tt|||||| S#t $rYXwxYw)zt Convert an HTTP date string (one of three formats) to seconds since epoch. @type dateString: C{bytes} rsSun, rr-EidilzUnknown datetime format %r:)r[rlowerweekdayname_lowerr6rlenisdigitfindintmonthname_lowerindexmapr5) dateStringpartspartlenrrrrrminsecs rr6r6ws  $ * * ,E 8Aa=   $5 5 #Hz$9: : %jG1 1 %(*:*:*< AhaQxQx Q,'Q,E!HMM#,>",D !8>>#.UDQx4y "9$;D CZ$;D UqAhaQxQx5 BCC c(C %%ekkm4 5E t9Djdjjo.ND#s vdE3c37 88Q   s F FFc6tt|dd|dfS)z Convert string to a chunk. @type data: C{bytes} @returns: a tuple of C{bytes} representing the chunked encoding of data xr)r\rdatas rr7r7s  SYqM +WdG DDrc&|D]}|dvsy|dk7S)z Is the string case-insensitively hexidecimal? It must be composed of one or more characters in the ranges a-f, A-F and 0-9. s0123456789abcdefABCDEFFrr)bcs r _ishexdigitsrs* - - 8OrcFt|s t|t|dS)z Decode a hexadecimal integer. Unlike L{int(b, 16)}, this raises L{ValueError} when the integer has a prefix like C{b'0x'}, C{b'+'}, or C{b'-'}, which is desirable when parsing network protocols. )rrr)rs r_hexintrs! ?m q":rc|jdd\}}t|}|dkrtd|fz|||dzdk7r td|d|||dzdfS)aA Convert chunk to string. Note that this function is not specification compliant: it doesn't handle chunk extensions. @type data: C{bytes} @return: tuple of (result, remaining) - both C{bytes}. @raise ValueError: If the given data is not a correctly formatted chunked byte string. rrrz!Chunk length must be >= 0, not %drzchunk must end with CRLFN)rrr)rprefixrestlengths rr8r8sv::gq)LFD V_F z<yHII FVaZ G+344 =$vz|, ,,rc|jj\}}|jdk7r td|jd\}}t t |jd\}}|dk(rd}n t |}|||fS)z Parse a content-range header into (start, end, realLength). realLength might be None if real length is not known ('*'). bytesz#a range of type %r is not supported/r*N)striprrrrr)rkindotherstartend realLengthstartends rr9r9s ,,.&&(KD% zz|w>?? ;;s+HjS(..-.JE3S _ 3 ##rcdeZdZdZedZedZdZdZdZ dZ dZ d d Z d d Z d Zy ))_IDeprecatedHTTPChannelToRequestInterfacez= The interface L{HTTPChannel} expects of L{Request}. zLA L{http_headers.Headers} instance giving all received HTTP request headers.zPA L{http_headers.Headers} instance holding all HTTP response headers to be sent.cy)z The underlying connection has been lost. @param reason: A failure instance indicating the reason why the connection was lost. @type reason: L{twisted.python.failure.Failure} Nr)reasons rconnectionLostz8_IDeprecatedHTTPChannelToRequestInterface.connectionLost rcy)a  Called when L{HTTPChannel} has determined the length, if any, of the incoming request's body. @param length: The length of the request's body. @type length: L{int} if the request declares its body's length and L{None} if it does not. Nrr s r gotLengthz3_IDeprecatedHTTPChannelToRequestInterface.gotLengthrrcy)z Deliver a received chunk of body data to the request. Note this does not imply chunked transfer encoding. @param data: The received chunk. @type data: L{bytes} Nrrs rhandleContentChunkz<_IDeprecatedHTTPChannelToRequestInterface.handleContentChunkrrcy)zF Parse the request's cookies out of received headers. Nrrrr parseCookiesz6_IDeprecatedHTTPChannelToRequestInterface.parseCookies'rrcy)a Called when the entire request, including its body, has been received. @param command: The request's HTTP command. @type command: L{bytes} @param path: The request's path. Note: this is actually what RFC7320 calls the URI. @type path: L{bytes} @param version: The request's HTTP version. @type version: L{bytes} Nr)commandrversions rrequestReceivedz9_IDeprecatedHTTPChannelToRequestInterface.requestReceived,rrcy)a Determines if two requests are the same object. @param other: Another object whose identity will be compared to this instance's. @return: L{True} when the two are the same object and L{False} when not. Nrrs r__eq__z0_IDeprecatedHTTPChannelToRequestInterface.__eq__<rrcy)a Determines if two requests are not the same object. @param other: Another object whose identity will be compared to this instance's. @return: L{True} when the two are not the same object and L{False} when they are. Nrr(s r__ne__z0_IDeprecatedHTTPChannelToRequestInterface.__ne__Grrcy)z{ Generate a hash value for the request. @return: The request's hash value. @rtype: L{int} Nrrrr__hash__z2_IDeprecatedHTTPChannelToRequestInterface.__hash__RrrNrobjectreturnbool)rrrrrLrequestHeadersresponseHeadersrrr r"r&r)r+r-rrrrrsQ N   O          rrc"eZdZdZdZdZdZy)r:zo I am a BytesIO wrapper that conforms for the transport API. I support the `writeSequence' method. c"t|_yN)rCrselfs r__init__zStringTransport.__init__as rcX|jjdj|y)Nr)rwritejoin)r8seqs r writeSequencezStringTransport.writeSequenceds  SXXc]#rc4t|jd|S)Nr)getattr__dict__)r8attrs r __getattr__zStringTransport.__getattr__gst}}S)400rN)rrrrr9r>rCrrrr:r:[s $1rr:cneZdZdZdZdZdZdZdZdZ dZ dZ d Z d Z d Zd Zd ZdZdZdZdZy)r;a@ A client for HTTP 1.0. Notes: You probably want to send a 'Host' header with the name of the site you're connecting to, in order to not break name based virtual hosting. @ivar length: The length of the request body in bytes. @type length: C{int} @ivar firstLine: Are we waiting for the first header line? @type firstLine: C{bool} @ivar __buffer: The buffer that stores the response to the HTTP request. @type __buffer: A C{BytesIO} object. @ivar _header: Part or all of an HTTP request header. @type _header: C{bytes} NTrcB|jj|d|dgy)Nrs HTTP/1.0  transportr>)r8r$rs r sendCommandzHTTPClient.sendCommands $$gtT;K%LMrct|tstt|}t |}t |}|j j |d|dgy)N: r)rr r\rrarGr>)r8rvalue santizedName santizedValues r sendHeaderzHTTPClient.sendHeadersI%'!#e*-E06 1%8  $$lE='%RSrc:|jjdy)NrrGr;r7s r endHeaderszHTTPClient.endHeaderss W%rc|jdd\}}|j}|j|||jdk(rt ||_yy)z Given a complete HTTP header, extract the field name and value and process the header. @param header: a complete HTTP request header of the form 'field-name: value'. @type header: C{bytes} :rcontent-lengthN)rlstrip handleHeaderrrr )r8rrvals r extractHeaderzHTTPClient.extractHeadersR<<a(Sjjl #s# 99;+ +c(DK ,rcJ|jr=d|_|jdd}|d}|d} |d}|j|||y|sZ|jdk7r|j |jt |_|j|jy|jds|jdr|j|z|_y|jr#|j |j||_y||_y#t$rd}YwxYw) z Parse the status line and headers for an HTTP request. @param line: Part of an HTTP request header. Request bodies are parsed in L{HTTPClient.rawDataReceived}. @type line: C{bytes} FNrrrr r) firstLiner IndexError handleStatus_headerrXrC_HTTPClient__bufferhandleEndHeaders setRawMode startswith)r8rlr%statusmessages r lineReceivedzHTTPClient.lineReceiveds >>"DN 4#AdGqTF A$   gvw 7 ||s"""4<<0#IDM  ! ! # OO   ??5 !T__T%: <<$.DL \\   t|| ,DLDL1  sD D"!D"c$|jyr6)handleResponseEnd)r8rs rrzHTTPClient.connectionLosts  rc|j3|jj}d|_|j|yy)z~ The response has been completely received. This callback may be invoked more than once per request. N)r_getvaluehandleResponse)r8rs rrhzHTTPClient.handleResponseEnds: == $ &&(A DM    " %rc:|jj|yr6)r_r;r8rs rhandleResponsePartzHTTPClient.handleResponseParts D!rcyr6rr7s rconnectionMadezHTTPClient.connectionMades rcy)z Called when the status-line is received. @param version: e.g. 'HTTP/1.0' @param status: e.g. '200' @type status: C{bytes} @param message: e.g. 'OK' Nr)r8r%rdres rr]zHTTPClient.handleStatusrrcy)z9 Called every time a header is received. Nr)r8rrWs rrVzHTTPClient.handleHeaderrrcy)z= Called when all headers have been received. Nrr7s rr`zHTTPClient.handleEndHeadersrrc|j=|d|j||jd}}|xjt|zc_nd}|j||jdk(r"|j|j |yy)Nrr)r rrnrh setLineMode)r8rr s rrawDataReceivedzHTTPClient.rawDataReceiveds{ ;; "m ,d4;;=.A$D KK3t9 $KD % ;;!   " " $   T " r)rrrrr r[r_r^rHrNrQrXrfrrhrnrpr]rVr`rvrrrr;r;ksa(FIHGNT& #' R! #"    #rr;)i0cL||dkr tStjS)zS Get a writeable file-like object to which request content can be written. i)rCtempfile TemporaryFilers r_getContentFiler{s'fvoy  ! ! ##rs^\[?(?P.*?)\]?(:\d+)?$ceZdZdZdZdZeZeeZ dZ dZ dZ dZ dZdZdZdZdZdZdZdZdZeZefd3dZd Zeed d d dd ZdZdZ dZ!dZ"d4dZ#dZ$dZ%dZ&d5dZ'dZ(d6dZ)dZ*dZ+ d7dZ,d8dZ-dZ.dZ/dZ0d Z1d!Z2d"Z3d#Z4d9d$Z5eed d%d&dd'(d)Z6d*Z7d+Z8d,Z9d-Z:d.Z;d/Zd2Z?y);r=a A HTTP request. Subclasses should override the process() method to determine how the request will be processed. @ivar method: The HTTP method that was used, e.g. C{b'GET'}. @type method: L{bytes} @ivar uri: The full encoded URI which was requested (including query arguments), e.g. C{b'/a/b%20/c?q=v'}. @type uri: L{bytes} @ivar path: The encoded path of the request URI (not including query arguments), e.g. C{b'/a/b%20/c'}. @type path: L{bytes} @ivar args: A mapping of decoded query argument names as L{bytes} to corresponding query argument values as L{list}s of L{bytes}. For example, for a URI with C{foo=bar&foo=baz&quux=spam} as its query part C{args} will be C{{b'foo': [b'bar', b'baz'], b'quux': [b'spam']}}. @type args: L{dict} of L{bytes} to L{list} of L{bytes} @ivar content: A file-like object giving the request body. This may be a file on disk, an L{io.BytesIO}, or some other type. The implementation is free to decide on a per-request basis. @type content: L{typing.BinaryIO} @ivar cookies: The cookies that will be sent in the response. @type cookies: L{list} of L{bytes} @type requestHeaders: L{http_headers.Headers} @ivar requestHeaders: All received HTTP request headers. @type responseHeaders: L{http_headers.Headers} @ivar responseHeaders: All HTTP response headers to be sent. @ivar notifications: A L{list} of L{Deferred}s which are waiting for notification that the response to this request has been finished (successfully or with an error). Don't use this attribute directly, instead use the L{Request.notifyFinish} method. @ivar _disconnected: A flag which is C{False} until the connection over which this request was received is closed and which is C{True} after that. @type _disconnected: L{bool} @ivar _log: A logger instance for request related messages. @type _log: L{twisted.logger.Logger} Nrs(no method yet)s(no clientproto yet)s (no uri yet)FcLg|_||_|jj|_|jj |_t |_i|_t |_ g|_ |jj|_ |turd}||_ y)z @param channel: the channel we're connected to. @param queued: (deprecated) are we in the request queue, or can we start writing to the transport? FN) notificationschannelgetPeerclientgetHosthostr`r2received_cookiesr3cookiesrG_QUEUED_SENTINELqueued)r8rrs rr9zRequest.__init__\s 46 ll**, LL((* '.y46(/ $& // % %F rc |jrK|jjdtt d|j |j |jj||`|j |jj|` |jD]}|jdg|_ y#t$rY7wxYw)zP Called when have finished responding and are no longer queued. z"Producer was not unregistered for N)producer_logfailurer_ RuntimeErroruriunregisterProducerr requestDonercloseOSErrorr~callback)r8rs r_cleanupzRequest._cleanupvs == II   'I$((%TUV   # # %   & L << #  ""$ ## A JJt     sC CCTwistedrrcy)a' Notify the object that it is no longer queued. We start writing whatever data we have to the transport, etc. This method is not intended for users. In 16.3 this method was changed to become a no-op, as L{Request} objects are now never queued. Nrr7s rnoLongerQueuedzRequest.noLongerQueueds rc$t||_y)a0 Called when HTTP channel got length of content in this request. This method is not intended for users. @param length: The length of the request body, as indicated by the request headers. L{None} if the request headers do not indicate a length. N)r{r)r8r s rrzRequest.gotLengths'v. rc|jjd}|y|D]P}|s|jdD]7}|j} |jdd\}}||j|<9Ry#t $rYHwxYw)zW Parse cookie headers. This method is not intended for users. scookieNrrr)r2 getRawHeadersrrUrr)r8 cookieheaders cookietxtcookrrs rr"zRequest.parseCookiess ++99)D  & I%OOD1D;;=D#zz$2134--a0  &s $A55 BBc:|jj|y)zX Write a chunk of data. This method is not intended for users. N)rr;rms rr zRequest.handleContentChunks 4 rc|jj}|jjddi|_||c|_|_||_|j jdd}t|dk(r|j |_ n|\|_ }t|d|_|j}|jjd}||d}|jdk(r|r|rd} t|} | dk(r4|jt|jjdn`| | k(r[ |jjd|jj} |jjt!| ||jjdd|j)y#t"$r|j$j'YywxYw) a Called by channel when all data has been received. This method is not intended for users. @type command: C{bytes} @param command: The HTTP verb of this request. This has the case supplied by the client (eg, it maybe "get" rather than "GET"). @type path: C{bytes} @param path: The URI of this request. @type version: C{bytes} @param version: The HTTP version of this request. r?rs content-typeNsPOSTsmultipart/form-datas!application/x-www-form-urlencoded)rtellseekargsmethodr clientprotorrrr2r2rrupdatereadrrr!_respondToBadRequestAndDisconnectprocess) r8r$rr%clengthr argstringrrmfdrrs rr&zRequest.requestReceiveds ,,##% !Q ' TX" HHNN4 # q6Q;DI#$ DIy A.DIyy##11/B  !HE ;;' !e(C#E*C:: HT\\%6%6%8!<=LL%%a("ll//1GII$$%6w%FG LL  a # 0LLBBDs8AF??#G%$G%c dj|jjt|t |j t |j t |jS)z Return a string description of the request including such information as the request method and request URI. @return: A string loosely describing this L{Request} object. @rtype: L{str} z.<{} at 0x{:x} method={} uri={} clientproto={}>)format __class__ridr[rrrr7s r__repr__zRequest.__repr__sR@FF NN # # tH  %  " )) *   rcy)zY Override in subclasses. This method is not intended for users. Nrr7s rrzRequest.process s rc|jrtd|d|jd||_||_|jj ||y)z& Register a producer. zregistering producer z before previous one (z) was unregisteredN)rrstreamingProducerrregisterProducerr8r streamings rrzRequest.registerProducersK =="*DMM;  "+   %%h :rcF|jjd|_y)z* Unregister the producer. N)rrrr7s rrzRequest.unregisterProducer%s '') rcH|jj|}||dSy)a Get an HTTP request header. @type key: C{bytes} or C{str} @param key: The name of the header to get the value of. @rtype: C{bytes} or C{str} or L{None} @return: The value of the specified header, or L{None} if that header was not present in the request. The string type of the result matches the type of C{key}. Nr)r2r)r8rrKs r getHeaderzRequest.getHeader.s-##11#6  9 rc8|jj|S)a. Get a cookie that was sent from the network. @type key: C{bytes} @param key: The name of the cookie to get. @rtype: C{bytes} or C{None} @returns: The value of the specified cookie, or L{None} if that cookie was not present in the request. )rget)r8rs r getCookiezRequest.getCookie?s$$((--rcf|jjt|jdS)a Notify when the response to this request has finished. @note: There are some caveats around the reliability of the delivery of this notification. 1. If this L{Request}'s channel is paused, the notification will not be delivered. This can happen in one of two ways; either you can call C{request.transport.pauseProducing} yourself, or, 2. In order to deliver this notification promptly when a client disconnects, the reactor must continue reading from the transport, so that it can tell when the underlying network connection has gone away. Twisted Web will only keep reading up until a finite (small) maximum buffer size before it gives up and pauses the transport itself. If this occurs, you will not discover that the connection has gone away until a timeout fires or until the application attempts to send some data via L{Request.write}. 3. It is theoretically impossible to distinguish between successfully I{sending} a response and the peer successfully I{receiving} it. There are several networking edge cases where the L{Deferred}s returned by C{notifyFinish} will indicate success, but the data will never be received. There are also edge cases where the connection will appear to fail, but in reality the response was delivered. As a result, the information provided by the result of the L{Deferred}s returned by this method should be treated as a guess; do not make critical decisions in your applications based upon it. @rtype: L{Deferred} @return: A L{Deferred} which will be triggered when the request is finished -- with a L{None} value if the request finishes successfully or with an error if the request is interrupted by an error (for example, the client closing the connection prematurely). r)r~rrUr7s r notifyFinishzRequest.notifyFinishLs,P !!(*-!!"%%rc|jr td|jrtjddy|j s|j d|jr|jj dt|jdr;|jj%|jjj|d |_|js|jyy) zV Indicate that all response data has been written to this L{Request}. zqRequest.finish called on a request after its connection was lost; use Request.notifyFinish to keep track of this.z%Warning! request.finish called twice.r) stacklevelNrs0 factoryr) _disconnectedrfinishedwarningswarnstartedWritingr;chunkedrhasattrrrZrrr7s rfinishzRequest.finishws   B  == MMAa P "" JJsO << LL  | , 4<< + 0D0D0P LL $ $T * {{ MMOrc|jr td|jry|jsd|_|j}d|j fz}|j }g}|dk(rT|jjd9|jdk7r*|j tvr|jdd|_ |jg|jjd r|jj!d n0|jj#d t%|jg|j&'|jj#d |j&g|jj)D]\}}|D]}|j||f!|j*D]} |jd | f|j,j/|||||jdk(r d |_y|j tvr d|_y|j2t5|z|_|rM|jr%|j,j7t9|y|j,j1|yy)z Write some data as a result of an HTTP request. The first time this is called, it writes out response data. @type data: C{bytes} @param data: Some bytes to be sent as part of the response body. zBRequest.write called on a request after Request.finish was called.Nrs%dHTTP/1.1rTHEAD)sTransfer-Encodingchunkeds last-modifiedzPWarning: last-modified specified both in header list and lastModified attribute.sETags Set-Cookiecyr6rrs rzRequest.write..rrcyr6rrs rrzRequest.write..rr)rrrrrcode code_messager3rrr<rr lastModified hasHeaderrinfo setRawHeadersr3etaggetAllRawHeadersrr writeHeadersr; sentLengthrr>r7) r8rr%rrheadersrvaluesrKcookies rr;z Request.writes: ==W     """#D &&GDII<'D&&FG K'))778IJRKK7*II]2AB    ,''112BCIINNC ((66(+;D._ensureBytess,{ #u% zz&))rc8t|jddS)z Replace linear whitespace (C{\r}, C{\n}, C{\r\n}) and semicolons C{;} in C{val} with a single space. @param val: L{bytes} @return: L{bytes} rr)rarrs r _sanitizez$Request.addCookie.._sanitize/s-S199$E ErrNs ; Expires=s ; Domain=s; Path=s ; Max-Age=s ; Comment=s; Secures ; HttpOnly)slaxsstrictzInvalid value for sameSite: s ; SameSite=)rrreprrr)r8rrexpiresdomainrmax_agecommentsecurehttpOnlysameSiterrrs r addCookiezRequest.addCookies;v *$ F<?+d2Y|A5OO  m+i W8M.NNF  l*Y|F7K-LLF  j(9\$5G+HHF  m+i W8M.NNF  m+i W8M.NNF k)F m+F #H-335H22 !?$x.!PQQ nx/ /F F#rct|ts td||_|r#t|ts td||_yt j|d|_y)zi Set the HTTP response code. @type code: L{int} @type message: L{bytes} z&HTTP response code must be int or longz*HTTP response status message must be bytessUnknown StatusN)rrrrr rr/r)r8rres rsetResponseCodezRequest.setResponseCodeOsV$$DE E gu- LMM 'D  ) d4E FD rc>|jj||gy)a Set an HTTP response header. Overrides any previously set values for this header. @type name: L{bytes} or L{str} @param name: The name of the header for which to set the value. @type value: L{bytes} or L{str} @param value: The value to set for the named header. A L{str} will be UTF-8 encoded, which may not interoperable with other implementations. Avoid passing non-ASCII characters if possible. N)r3r)r8rrKs r setHeaderzRequest.setHeader`s **4%9rcR|jt|jd|y)a- Utility function that does a redirect. Set the response code to L{FOUND} and the I{Location} header to the given URL. The request should have C{finish()} called after this. @param url: I{Location} header value. @type url: L{bytes} or L{str} slocationN)rrr)r8rs rredirectzRequest.redirectos  U# {C(rc^ttj|}|jr|j|kr||_|j d}|rK|j ddd} t |}||jk\r|jttSy#t$rYywxYw)a Set the C{Last-Modified} time for the response to this request. If I am called more than once, I ignore attempts to set Last-Modified earlier, only replacing the Last-Modified time if it is to a later value. If I am a conditional request, I may modify my response code to L{NOT_MODIFIED} if appropriate for the time given. @param when: The last time the resource being returned was modified, in seconds since the epoch. @type when: number @return: If I am a I{If-Modified-Since} conditional request and the time given is not newer than the condition, I return L{http.CACHED} to indicate that you should write no body. Otherwise, I return a false value. sif-modified-sincerrrN) rmathceilrrrr6rrrr0)r8when modifiedSince firstParts rsetLastModifiedzRequest.setLastModified~s*499T?#!!t'8'84'? $D ';< %++D!4Q7I  0 ;  1 11$$\2    s* B B,+B,c|r||_|jd}|rK|j}||vsd|vr3|j|jdvxrt xst tSy)a Set an C{entity tag} for the outgoing response. That's "entity tag" as in the HTTP/1.1 C{ETag} header, "used for comparing two or more entities from the same requested resource." If I am a conditional request, I may modify my response code to L{NOT_MODIFIED} or L{PRECONDITION_FAILED}, if appropriate for the tag given. @param etag: The entity tag for the resource being returned. @type etag: string @return: If I am a C{If-None-Match} conditional request and the tag matches one in the request, I return L{http.CACHED} to indicate that you should write no body. Otherwise, I return a false value. s if-none-match*)rsGETN)rrrrrrr!r0)r8rtagss rsetETagzRequest.setETagsj& DI~~./ ::>$ 40 ?!!rc |jd}|sdx|_|_y|j\}}|j dk7r t t j|}|jdd\|_|_y#tjt f$rdx|_|_Yyt$r,|jjddx|_|_YywxYw)Ns AuthorizationrsbasicrSrr) ruserpasswordrrrbase64 b64decodebinasciiError BaseExceptionrr)r8authhbasupws r _authorizezRequest._authorizeBs ,NN#34E,// DM{{}HCyy{h& l"""3'C'*yyq'9 $DIt} + ,(+ +DI  , II  b !(+ +DI  ,s!B A$B 'C&21C&%C&cr |jS#t$rYnwxYw|j|jS)z Return the HTTP user sent with this request, if any. If no user was supplied, return the empty string. @returns: the HTTP user, if any @rtype: C{bytes} )rr#r'r7s rgetUserzRequest.getUserTs7 99     yy  cr |jS#t$rYnwxYw|j|jS)z Return the HTTP password sent with this request, if any. If no password was supplied, return the empty string. @returns: the HTTP password, if any @rtype: C{bytes} )rr#r'r7s r getPasswordzRequest.getPasswordds7 ==     }}r*cd|_d|_|j|jj|jD]}|j |g|_y)z There is no longer a connection for this request to respond over. Clean up anything which can't be useful anymore. TN)rrrrr~errback)r8rrs rrzRequest.connectionLosttsV " << # LL   ## A IIf  rcR|j|jjyy)zL Pass the loseConnection through to the underlying channel. N)rloseConnectionr7s rr0zRequest.loseConnections# << # LL ' ' ) $rc6t|tr||uStS)a Determines if two requests are the same object. @param other: Another object whose identity will be compared to this instance's. @return: L{True} when the two are the same object and L{False} when not. @rtype: L{bool} )rr=NotImplemented)r8rs rr)zRequest.__eq__s eW %5= rct|S)z A C{Request} is hashable so that it can be used as a mapping key. @return: A C{int} based on the instance's identity. )rr7s rr-zRequest.__hash__s $xr)rr?rr/r0None)r0r)rrDr0zOptional[AnyStr])r0zDeferred[None])NNNNNNFNr6)rr.)@rrrrrrrrr/rrrrrrrrrrrrr rrWrrr9rr^rPrrr"r r&rrrrrrrrr;rrrrrrrrrrrrr r'r)r,rr0r)r-rrrr=r=s2hHH DR=L F)K CNGJ DL D DGIM 8D>N4 2 2q!,-  .   /*!5n   ;" .)&V:L)d k$ZG" : )#JB 2" ;D 2q!,:LM N  "*,$   *(rr=ceZdZdZy) _DataLossa; L{_DataLoss} indicates that not all of a message body was received. This is only one of several possible exceptions which may indicate that data was lost. Because of this, it should not be checked for by specifically; any unexpected exception should be treated as having caused data loss. Nrrrrr6r6srr6ceZdZdZy)r>a L{PotentialDataLoss} may be raised by a transfer encoding decoder's C{noMoreData} method to indicate that it cannot be determined if the entire response body has been delivered. This only occurs when making requests to HTTP servers which do not set I{Content-Length} or a I{Transfer-Encoding} in the response because in this case the end of the response is indicated by the connection being closed, an event which may also be due to a transient network problem or other error. Nrrrrr>r>srr>ceZdZdZy)_MalformedChunkedDataErrora C{_ChunkedTransferDecoder} raises L{_MalformedChunkedDataError} from its C{dataReceived} method when it encounters malformed data. This exception indicates a client-side error. If this exception is raised, the connection should be dropped with a 400 error. Nrrrrr9r9srr9c"eZdZdZdZdZdZy)_IdentityTransferDecoderaZ Protocol for accumulating bytes up to a specified length. This handles the case where no I{Transfer-Encoding} is specified. @ivar contentLength: Counter keeping track of how many more bytes there are to receive. @ivar dataCallback: A one-argument callable which will be invoked each time application data is received. @ivar finishCallback: A one-argument callable which will be invoked when the terminal chunk is received. It will be invoked with all bytes which were delivered to this protocol which came after the terminal chunk. c.||_||_||_yr6) contentLength dataCallbackfinishCallback)r8r=r>r?s rr9z!_IdentityTransferDecoder.__init__s*(,rc|j td|j|j|yt||jkr0|xjt|zc_|j|y|j}|j}|j}dx|_|_d|_||d||||dy)a; Interpret the next chunk of bytes received. Either deliver them to the data callback or invoke the finish callback if enough bytes have been received. @raise RuntimeError: If the finish callback has already been invoked during a previous call to this methood. Nz;_IdentityTransferDecoder cannot decode data after finishingr)r>rr=rr?)r8rr=r>r?s r dataReceivedz%_IdentityTransferDecoder.dataReceiveds    $M     %   d # Y++ +   #d) +    d # !..M,,L!00N6: :D  3!"D  n}- . 4 / 0rc|j}dx|_|_|j|dt|jdk7r t y)ap All data which will be delivered to this decoder has been. Check to make sure as much data as was expected has been received. @raise PotentialDataLoss: If the content length is unknown. @raise _DataLoss: If the content length is known and fewer than that many bytes have been delivered. @return: L{None} Nrr)r?r>r=r>r6)r8r?s r noMoreDataz#_IdentityTransferDecoder.noMoreDatasW,,266D/    % 3 #% %   1 $+ %rN)rrrrr9rArCrrrr;r;s - 1@rr;is !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[]^_`abcdefghijklmnopqrstuvwxyz{|}~c`eZdZdZdZ d dZd dZd dZd dZd dZ d dZ dd Z dd Z y )_ChunkedTransferDecoderai Protocol for decoding I{chunked} Transfer-Encoding, as defined by RFC 7230, section 4.1. This protocol can interpret the contents of a request or response body which uses the I{chunked} Transfer-Encoding. It cannot interpret any of the rest of the HTTP protocol. It may make sense for _ChunkedTransferDecoder to be an actual IProtocol implementation. Currently, the only user of this class will only ever call dataReceived on it. However, it might be an improvement if the user could connect this to a transport and deliver connection lost notification. This way, `dataCallback` becomes `self.transport.write` and perhaps `finishCallback` becomes `self.transport.loseConnection()` (although I'm not sure where the extra data goes in that case). This could also allow this object to indicate to the receiver of data that the stream was not completely received, an error case which should be noticed. -exarkun @ivar dataCallback: A one-argument callable which will be invoked each time application data is received. This callback is not reentrant. @ivar finishCallback: A one-argument callable which will be invoked when the terminal chunk is received. It will be invoked with all bytes which were delivered to this protocol which came after the terminal chunk. @ivar length: Counter keeping track of how many more bytes in a chunk there are to receive. @ivar state: One of C{'CHUNK_LENGTH'}, C{'CRLF'}, C{'TRAILER'}, C{'BODY'}, or C{'FINISHED'}. For C{'CHUNK_LENGTH'}, data for the chunk length line is currently being read. For C{'CRLF'}, the CR LF pair which follows each chunk is being read. For C{'TRAILER'}, the CR LF pair which follows the terminal 0-length chunk is currently being read. For C{'BODY'}, the contents of a chunk are being read. For C{'FINISHED'}, the last chunk has been completely read and no more input is valid. @ivar _buffer: Accumulated received data for the current state. At each state transition this is truncated at the front so that index 0 is where the next state shall begin. @ivar _start: While in the C{'CHUNK_LENGTH'} and C{'TRAILER'} states, tracks the index into the buffer at which search for CRLF should resume. Resuming the search at this position avoids doing quadratic work if the chunk length line arrives over many calls to C{dataReceived}. @ivar _trailerHeaders: Accumulates raw/unparsed trailer headers. See https://github.com/twisted/twisted/issues/12014 @ivar _maxTrailerHeadersSize: Maximum bytes for trailer header from the response. @type _maxTrailerHeadersSize: C{int} @ivar _receivedTrailerHeadersSize: Bytes received so far for the tailer headers. @type _receivedTrailerHeadersSize: C{int} CHUNK_LENGTHcv||_||_t|_d|_g|_d|_d|_y)Nri)r>r? bytearray_buffer_start_trailerHeaders_maxTrailerHeadersSize_receivedTrailerHeadersSize)r8r>r?s rr9z _ChunkedTransferDecoder.__init__us< ), {  02&+#+,(rc|jjd|j}|tk\s!|dk(r:t |jtkDrt dj t|dk(rt |jdz |_y|jjdd|}|dk(r|}|jd|} t|}|j|dz|}|r(|jd td k7rt d |d |dk(rd |_ nd|_ ||_ |jd|dz=d|_y#t$r t dwxYw)ak Read the chunk size line, ignoring any extensions. @returns: C{True} once the line has been read and removed from C{self._buffer}. C{False} when more data is required. @raises _MalformedChunkedDataError: when the chunk size cannot be decoded or the length of the line exceeds L{maxChunkSizeLineLength}. rrz,Chunk size line exceeds maximum of {} bytes.rFrrzChunk-size must be an integer.Nrz(Invalid characters in chunk extensions: .TRAILERBODYrT) rIrrJmaxChunkSizeLineLengthrr9rrr translate_chunkExtCharsstater )r8eolIndexendOfLengthIndex rawLengthr exts r_dataReceived_CHUNK_LENGTHz2_ChunkedTransferDecoder._dataReceived_CHUNK_LENGTHsj<<$$Wdkk: - - Ns4<<03II,>EE*  r>dll+a/DK<<,,T1h? r !' LL#34  OY'Fll+a/(; 3==~6#=,:3'C  Q;"DJDJ LLX\) * # O,-MN N Os  EEct|jdkry|jjds tdd|_|jdd=y)a; Await the carriage return and line feed characters that are the end of chunk marker that follow the chunk data. @returns: C{True} when the CRLF have been read, otherwise C{False}. @raises _MalformedChunkedDataError: when anything other than CRLF are received. rFrzChunk did not end with CRLFrFrT)rrIrbr9rUr7s r_dataReceived_CRLFz*_ChunkedTransferDecoder._dataReceived_CRLFsP t|| q ||&&w/,-JK K# LL1 rc|jjd|j}|dk(r]|jt |jz|jj drdndz}||j kDr tdy|dkDr~|jj|jd||jd|dz=d|_|xj|dzz c_|j|j kDr tdy t|jdd j}|jd d =d |_ |j|y) a Collect trailer headers if received and finish at the terminal zero-length chunk. Then invoke C{finishCallback} and switch to state C{'FINISHED'}. @returns: C{False}, as there is either insufficient data to continue, or no data remains. rr rrz!Trailer headers data is too long.FrTNFINISHED)rIrrJrMrendswithrLr9rKr memoryviewtobytesrUr?)r8rVminTrailerSizers r_dataReceived_TRAILERz-_ChunkedTransferDecoder._dataReceived_TRAILERs=<<$$Wdkk: r>00dll#$ --e41!=   ; ;;01TUU a<  ' ' Qx(@ A QA-.DK  , ,1 < ,//$2M2MM01TUU$,,'+335 LLO  D!rct|j|jk\rbt|jd|jj }|jd|j=d|_|j |yt|j}|xjt|zc_|jdd=|j |y)z Deliver any available chunk data to the C{dataCallback}. When all the remaining data for the chunk arrives, switch to state C{'CRLF'}. @returns: C{True} to continue processing of any buffered data. NCRLFT)rrIr rarbrUr>r )r8chunks r_dataReceived_BODYz*_ChunkedTransferDecoder._dataReceived_BODYs t||  +t||,]t{{;CCEE ]t{{]+DJ   e $  $,,'E KK3u: %K Q   e $rctd)z Once C{finishCallback} has been invoked receipt of additional data raises L{RuntimeError} because it represents a programming error in the caller. zJ_ChunkedTransferDecoder.dataReceived called after last chunk was processed)rr7s r_dataReceived_FINISHEDz._ChunkedTransferDecoder._dataReceived_FINISHED s  "  rc|xj|z c_d}|r<|jr/t|d|jz}|r|jr-yyyy)zu Interpret data from a request or response body which uses the I{chunked} Transfer-Encoding. T_dataReceived_N)rIr@rU)r8rgoOns rrAz$_ChunkedTransferDecoder.dataReceivedsL  t||?74!1DJJ!>?ADt||d|drcT|jdk7rtd|jdy)zj Verify that all data has been received. If it has not been, raise L{_DataLoss}. r_zChunked decoder in z= state, still expecting more data to get to 'FINISHED' state.N)rUr6r7s rrCz"_ChunkedTransferDecoder.noMoreData s. :: #.2jj;  $rN)r>Callable[[bytes], None]r?ror0r4)r0r1rr r0r4)r0r4) rrrrrUr9rZr\rdrhrjrArCrrrrErE9sV7r E -- -0 -  -3j(+Z&  B rrEc.eZdZdZdZdZdZdZdZy)_NoPushProducerz A no-op version of L{interfaces.IPushProducer}, used to abstract over the possibility that a L{HTTPChannel} transport does not provide L{IPushProducer}. cy)z Pause producing data. Tells a producer that it has produced too much data to process for the time being, and to stop until resumeProducing() is called. Nrr7s rpauseProducingz_NoPushProducer.pauseProducing4rrcy)z Resume producing data. This tells a producer to re-add itself to the main loop and produce more data for its consumer. Nrr7s rresumeProducingz_NoPushProducer.resumeProducing<rrcy)z Register to receive data from a producer. @param producer: The producer to register. @param streaming: Whether this is a streaming producer or not. Nrrs rrz _NoPushProducer.registerProducerDrrcy)zM Stop consuming data from a producer, without disconnecting. Nrr7s rrz"_NoPushProducer.unregisterProducerLrrcy)z) IProducer.stopProducing Nrr7s r stopProducingz_NoPushProducer.stopProducingQrrN) rrrrrtrvrrrzrrrrrrr,s       rrrc eZdZdZdZdZdZdZdZdZ dZ dZ e Z dZdZdZdZdZd ZdZdZeZd Zd Zd Zd ZdZdZdZd&dZdZ dZ!dZ"dZ#dZ$dZ%dZ&dZ'dZ(dZ)dZ*dZ+dZ,dZ-d Z.d!Z/d"Z0d#Z1d$Z2d%Z3y)'r?a A receiver for HTTP requests. The L{HTTPChannel} provides L{interfaces.ITransport} and L{interfaces.IConsumer} to the L{Request} objects it creates. It also implements L{interfaces.IPushProducer} to C{self.transport}, allowing the transport to pause it. @ivar MAX_LENGTH: Maximum length for initial request line and each line from the header. @ivar _transferDecoder: L{None} or a decoder instance if the request body uses the I{chunked} Transfer-Encoding. @type _transferDecoder: L{_ChunkedTransferDecoder} @ivar maxHeaders: Maximum number of headers allowed per request. @type maxHeaders: C{int} @ivar totalHeadersSize: Maximum bytes for request line plus all headers from the request. @type totalHeadersSize: C{int} @ivar _receivedHeaderSize: Bytes received so far for the header. @type _receivedHeaderSize: C{int} @ivar _handlingRequest: Whether a request is currently being processed. @type _handlingRequest: L{bool} @ivar _dataBuffer: Any data that has been received from the connection while processing an outstanding request. @type _dataBuffer: L{list} of L{bytes} @ivar _networkProducer: Either the transport, if it provides L{interfaces.IPushProducer}, or a null implementation of L{interfaces.IPushProducer}. Used to attempt to prevent the transport from producing excess data when we're responding to a request. @type _networkProducer: L{interfaces.IPushProducer} @ivar _requestProducer: If the L{Request} object or anything it calls registers itself as an L{interfaces.IProducer}, it will be stored here. This is used to create a producing pipeline: pause/resume producing methods will be propagated from the C{transport}, through the L{HTTPChannel} instance, to the c{_requestProducer}. The reason we proxy through the producing methods rather than the old behaviour (where we literally just set the L{Request} object as the producer on the transport) is because we want to be able to exert backpressure on the client to prevent it from sending in arbitrarily many requests without ever reading responses. Essentially, if the client never reads our responses we will eventually stop reading its requests. @type _requestProducer: L{interfaces.IPushProducer} @ivar _requestProducerStreaming: A boolean that tracks whether the producer on the L{Request} side of this channel has registered itself as a L{interfaces.IPushProducer} or an L{interfaces.IPullProducer}. @type _requestProducerStreaming: L{bool} or L{None} @ivar _waitingForTransport: A boolean that tracks whether the transport has asked us to stop producing. This is used to keep track of what we're waiting for: if the transport has asked us to stop producing then we don't want to unpause the transport until it asks us to produce again. @type _waitingForTransport: L{bool} @ivar abortTimeout: The number of seconds to wait after we attempt to shut the transport down cleanly to give up and forcibly terminate it. This is only used when we time a connection out, to prevent errors causing the FD to get leaked. If this is L{None}, we will wait forever. @type abortTimeout: L{int} @ivar _abortingCall: The L{twisted.internet.base.DelayedCall} that will be used to forcibly close the transport if it doesn't close cleanly. @type _abortingCall: L{twisted.internet.base.DelayedCall} @ivar _optimisticEagerReadSize: When a resource takes a long time to answer a request (via L{twisted.web.server.NOT_DONE_YET}, hopefully one day by a L{Deferred}), we would like to be able to let that resource know about the underlying transport disappearing as promptly as possible, via L{Request.notifyFinish}, and therefore via C{self.requests[...].connectionLost()} on this L{HTTPChannel}. However, in order to simplify application logic, we implement head-of-line blocking, and do not relay pipelined requests to the application until the previous request has been answered. This means that said application cannot dispose of any entity-body that comes in from those subsequent requests, which may be arbitrarily large, and it may need to be buffered in memory. To implement this tradeoff between prompt notification when possible (in the most frequent case of non-pipelined requests) and correct behavior when not (say, if a client sends a very long-running GET request followed by a PUT request with a very large body) we will continue reading pipelined requests into C{self._dataBuffer} up to a given limit. C{_optimisticEagerReadSize} is the number of bytes we will accept from the client and buffer before pausing the transport. This behavior has been in place since Twisted 17.9.0 . @type _optimisticEagerReadSize: L{int} ii@rrrNFc<g|_d|_g|_d|_yNF)requests_handlingRequest _dataBuffer_transferDecoderr7s rr9zHTTPChannel.__init__s! % $rc|j|jtj|jt |_|j j|dy)NT) setTimeouttimeOutrR IPushProducerrGrr_networkProducerrr7s rrpzHTTPChannel.connectionMadesI  % * 8 8 NNO-!  ..tT:rc|j|xjt|z c_|j|jkDr|j y|j r |j sdx|_|_y|s|j dk(rd|_ytj|jr|j|}n%|j|t|j}|jj|d|_|j}t|dk7r|j y|\}}} |jd||_||_||_y|dk(rr|j(r|j+|j(}|syd|_|j-|j.dk(r|j1y|j3y|dd vr(|xj(d |j5d zz c_y|j(r|j+|j(||_y#t $r|j YywxYw) zp Called for each line from request until the end of headers when it enters binary mode. Ncyr6r)rs rrz*HTTPChannel.lineReceived..rrrrrrrr r) resetTimeout_receivedHeaderSizertotalHeadersSizer_HTTPChannel__first_line persistentrArfrc providedByrequestFactoryrrrr|UnicodeDecodeError_command_path_version_HTTPChannel__headerheaderReceivedallHeadersReceivedr allContentReceivedrarU)r8rrequestrr$r%oks rrfzHTTPChannel.lineReceiveds    CI-  # #d&;&; ;  2 2 4    ??8JJ!D$5D--2$%!(2243F3FG--d3--dC 4FG MM  ) !D JJLE5zQ668(- %GWg w' $DM DJ#DM S[}}((7DM  # # %{{a'')! !W  MMTDKK$77 7M }}##DMM2 DM=& 668 s II! I!cZ|jj||jyr6)rrrrms r_finishRequestBodyzHTTPChannel._finishRequestBody/ s" % !rcfd}|dk(rR|js|S t|}t|jdj j }nd|dk(r^|jdk(r0d}tjdj j }n|jdk(ry|Syj|S|_ |_ y#t$r |cYSwxYw) z If the provided header is C{content-length} or C{transfer-encoding}, choose the appropriate decoder if any. Returns L{True} if the request can proceed and L{False} if not. c4jd_yr~)rr r7srfailz5HTTPChannel._maybeChooseTransferDecoder..fail; s  2 2 4DKrrTrstransfer-encodingrNsidentityT) rrrr;rr rrrErr )r8rrrr newTransferDecoders` r_maybeChooseTransferDecoderz'HTTPChannel._maybeChooseTransferDecoder3 s  & &<<>v  T": b)<U>U" + +zz|z)%<MM"%88$:Q:Q&",v   ,6M DK$6D !5 v  s C$$C76C7cB |jdd\}}|r|ddjr|jy|j }|j d}|j ||sy|jdj}|j|}||j|n|j||g|xjdz c_ |j|jkDr|jyy#t$r|jYywxYw)af Do pre-processing (for content-length) and store this header away. Enforce the per-request header limit. @type line: C{bytes} @param line: A line from the header section of a request, excluding the line delimiter. @return: A flag indicating whether the header was valid. @rtype: L{bool} rSrFrNrT)rrrisspacerrrrr2rrr_receivedHeaderCount maxHeaders)r8rrr reqHeadersrs rrzHTTPChannel.headerReceivedb s  ::dA.LFD ,,.  2 2 4zz&!//=]]2&55 ))&1   MM$   $ $VdV 4 !!Q&!  $ $t 6  2 2 45   2 2 4 sDDDcT|j}|j}|j}d|_d|_d|_d|_d|_|`|`|`|jr|jd|_ d|_ |j|jd}|j|||y)NrrTr)rrrr rrrrrr _savedTimeOutrrarr&)r8r$rr%reqs rrzHTTPChannel.allContentReceived s--zz-- $%!#$  $ M4:t} <>;Q;Q;S7T  " &&(rc|jd|jD]}|j||j"|jj d|_yyr6)rrrrcancel)r8rrs rrzHTTPChannel.connectionLost? s\ }} +G  " "6 * +    )    % % '!%D  *rcFtj|jdyy)a# Return L{True} if this channel is using a secure transport. Normally this method returns L{True} if this instance is using a transport that implements L{interfaces.ISSLTransport}. @returns: L{True} if this request is secure @rtype: C{bool} NTF)rR ISSLTransportrGr7s rr zHTTPChannel.isSecureI s!  # #DNND 9 Erct}|D]\}}|j|||dz|zdz|zdz}|g} | jd|jD| j d|j j | y)a Called by L{Request} objects to write a complete set of HTTP headers to a transport. @param version: The HTTP version in use. @type version: L{bytes} @param code: The HTTP status code to write. @type code: L{bytes} @param reason: The HTTP reason phrase to write. @type reason: L{bytes} @param headers: The headers to write to the transport. @type headers: L{twisted.web.http_headers.Headers} rrc3FK|]\}}|D]}|dz|zdzyw)rJrNr).0rrrKs r z+HTTPChannel.writeHeaders..n s? f  5L5 7 * * s!N)r` addRawHeaderextendrrrGr>) r8r%rrrsanitizedHeadersrrK responseLineheaderSequences rrzHTTPChannel.writeHeadersW s"#9" 7KD%  ) )$ 6 7~,t3fzHTTPChannel.writeSequence s $$U+rc6|jjS)zg Get the remote address of this connection. @return: An L{IAddress} provider. )rGrr7s rrzHTTPChannel.getPeer  ~~%%''rc6|jjS)zf Get the local address of this connection. @return: An L{IAddress} provider. )rGrr7s rrzHTTPChannel.getHost rrcj|jj|jjS)z Closes the connection. Will write any data that is pending to be sent on the network, but if this response has not yet been written to the network will not write anything. @return: L{None} )rrrGr0r7s rr0zHTTPChannel.loseConnection s) 002~~,,..rc|jtd|d|jd|s t||}||_||_|s|j yy)a Register to receive data from a producer. This sets self to be a consumer for a producer. When this object runs out of data (as when a send(2) call on a socket succeeds in moving the last data from a userspace buffer into a kernelspace buffer), it will ask the producer to resumeProducing(). For L{IPullProducer} providers, C{resumeProducing} will be called once each time data is required. For L{IPushProducer} providers, C{pauseProducing} will be called whenever the write buffer fills up and C{resumeProducing} will only be called when it empties. @type producer: L{IProducer} provider @param producer: The L{IProducer} that will be producing data. @type streaming: L{bool} @param streaming: C{True} if C{producer} provides L{IPushProducer}, C{False} if C{producer} provides L{IPullProducer}. @raise RuntimeError: If a producer is already registered. @return: L{None} NzCannot register producer z, because producer z was never unregistered.)_requestProducerrrT_requestProducerStreamingstartStreamingrs rrzHTTPChannel.registerProducer sc6  ,#+T-B-BD  "8T2H ()2&  # # %rc|jy|js|jjd|_d|_y)zg Stop consuming data from a producer, without disconnecting. @return: L{None} N)rr stopStreamingr7s rrzHTTPChannel.unregisterProducer s>  ( --  ! ! / / 1 $)-&rcR|j|jjyy)a& Stop producing data. The HTTPChannel doesn't *actually* implement this, beacuse the assumption is that it will only be called just before C{loseConnection} is called. There's nothing sensible we can do other than call C{loseConnection} anyway. N)rrzr7s rrzzHTTPChannel.stopProducing s'  ,  ! ! / / 1 -rcd|_|j|jj|js|jjyy)a" Pause producing data. This will be called by the transport when the send buffers have been filled up. We want to simultaneously pause the producing L{Request} object and also pause our transport. The logic behind pausing the transport is specifically to avoid issues like https://twistedmatrix.com/trac/ticket/8868. In this case, our inability to send does not prevent us handling more requests, which means we increasingly queue up more responses in our send buffer without end. The easiest way to handle this is to ensure that if we are unable to send our responses, we will not read further data from the connection until the client pulls some data out. This is a bit of a blunt instrument, but it's ok. Note that this potentially interacts with timeout handling in a positive way. Once the transport is paused the client may run into a timeout which will cause us to tear the connection down. That's a good thing! TN)rrrtrrr7s rrtzHTTPChannel.pauseProducing sN,%)!  ,  ! ! 0 0 2$$  ! ! 0 0 2%rcd|_|j|jj|js|jjyy)a! Resume producing data. This will be called by the transport when the send buffer has dropped enough to actually send more data. When this happens we can unpause any outstanding L{Request} producers we have, and also unpause our transport. FN)rrrvrrr7s rrvzHTTPChannel.resumeProducing sL%*!  ,  ! ! 1 1 3$$  ! ! 1 1 3%rc:|jjdy)z} Sends a 100 Continue response, used to signal to clients that further processing will be performed. sHTTP/1.1 100 Continue NrPr7s rrzHTTPChannel._send100Continue" s =>rcZ|jjd|jy)a& This is a quick and dirty way of responding to bad requests. As described by HTTP standard we should be patient and accept the whole request from the client before sending a polite bad request response, even in the case when clients send tons of data. sHTTP/1.1 400 Bad Request N)rGr;r0r7s rrz-HTTPChannel._respondToBadRequestAndDisconnect) s# @A rrp)4rrrrrrrr rrr_HTTPChannel__contentr=rrrrrrrrrrWrr9rprfrrrrrvrrrrrrr rr;r>rrr0rrrzrtrvrrrrrr?r?WsfPJL FJHLINM $ M% 8D%;G!R"-^(T48)5V $)V"4 )& 5> # ,(( /(&T . 2!3F4&? rr?c>t|ts|jd}t|}t|ts|j d}|j dr|dd}|j dr%|ddjddjd dS|ddS) z Return a string like python repr, but always escaped as if surrounding quotes were double quotes. @param s: The string to escape. @type s: L{bytes} or L{str} @return: An escaped string. @rtype: L{str} rrrN'r"z\"z\')rr r~rrr|rbr)rrs r_escaper5 s a  HHW  QA a  HHW ||C abE||C2wsE*225#>> Qr7Nrc |j}t|tjtjt fr |j }nd}t|jdxsd}t|jdxsd}dtt||t|jt|jt|j|j|jxsd|| z}|S)zm @return: A combined log formatted log line for the given request. @see: L{IAccessLogFormatter} -sreferers user-agentzk"%(ip)s" - - %(timestamp)s "%(method)s %(uri)s %(protocol)s" %(code)d %(length)s "%(referrer)s" "%(agent)s"r) ip timestamprrrSrr referreragent)rrrQrr_XForwardedForAddressrrrdictrrrrr)rr clientAddrrrrrs rcombinedLogFormatterrM s))+JW(('*=*=?TU__ w((4<=H G%%m4< =E 9 r{7>>* $W001%%,    KrceZdZdZdZy)rz L{IAddress} which represents the client IP to log for a request, as gleaned from an X-Forwarded-For header. @ivar host: An IP address or C{b"-"}. @type host: L{bytes} @see: L{proxiedLogFormatter} c||_yr6rr8rs rr9z_XForwardedForAddress.__init__{ s  rN)rrrrr9rrrrro s rrcFeZdZdZdZedZedZedZy)_XForwardedForRequestz Add a layer on top of another request that only uses the value of an X-Forwarded-For header as the result of C{getClientAddress}. c|jjjddgdjddj }t |S)a8 The client address (the first address) in the value of the I{X-Forwarded-For header}. If the header is not present, the IP is considered to be C{b"-"}. @return: L{_XForwardedForAddress} which wraps the client address as expected by L{combinedLogFormatter}. sx-forwarded-forrr,)_requestr2rrrrrs rrz&_XForwardedForRequest.getClientAddress sP MM ( ( 6 67ID6 RST U U4[  UW %T**rc.|jjS)zX @return: The protocol version in the request. @rtype: L{bytes} )rrr7s rrz!_XForwardedForRequest.clientproto s }}(((rc.|jjS)zT @return: The response code for the request. @rtype: L{int} )rrr7s rrz_XForwardedForRequest.code s }}!!!rc.|jjS)z` @return: The number of bytes sent in the response body. @rtype: L{int} )rrr7s rrz _XForwardedForRequest.sentLength s }}'''rN) rrrrrpropertyrrrrrrrr sH +"))""((rrrc,t|t|S)z @return: A combined log formatted log line for the given request but use the value of the I{X-Forwarded-For} header as the value for the client IP address. @see: L{IAccessLogFormatter} )rr)rrs rproxiedLogFormatterr s +@+I JJrc2eZdZdZdZeZdZdZdZ dZ e dZ e jdZ e dZejdZe dZejdZe d Zejd Ze d Zejd Zd Zy)_GenericHTTPChannelProtocola A proxy object that wraps one of the HTTP protocol objects, and switches between them depending on TLS negotiated protocol. @ivar _negotiatedProtocol: The protocol negotiated with ALPN or NPN, if any. @type _negotiatedProtocol: Either a bytestring containing the ALPN token for the negotiated protocol, or L{None} if no protocol has yet been negotiated. @ivar _channel: The object capable of behaving like a L{HTTPChannel} that is backing this object. By default this is a L{HTTPChannel}, but if a HTTP protocol upgrade takes place this may be a different channel object. Must implement L{IProtocol}. @type _channel: L{HTTPChannel} @ivar _requestFactory: A callable to use to build L{IRequest} objects. @type _requestFactory: L{IRequest} @ivar _site: A reference to the creating L{twisted.web.server.Site} object. @type _site: L{twisted.web.server.Site} @ivar _factory: A reference to the creating L{HTTPFactory} object. @type _factory: L{HTTPFactory} @ivar _timeOut: A timeout value to pass to the backing channel. @type _timeOut: L{int} or L{None} @ivar _callLater: A value for the C{callLater} callback. @type _callLater: L{callable} Nc.|jjS)z= @see: L{_genericHTTPChannelProtocolFactory} )_channelrr7s rrz#_GenericHTTPChannelProtocol.factory  }}$$$rc4||_||j_yr6)_factoryrrr8rKs rrz#_GenericHTTPChannelProtocol.factory s % rc.|jjS)z A callable to use to build L{IRequest} objects. Retries the object from the current backing channel. )rrr7s rrz*_GenericHTTPChannelProtocol.requestFactory s}}+++rc4||_||j_y)a  A callable to use to build L{IRequest} objects. Sets the object on the backing channel and also stores the value for propagation to any new channel. @param value: The new callable to use. @type value: A L{callable} returning L{IRequest} N)_requestFactoryrrrs rrz*_GenericHTTPChannelProtocol.requestFactory s %', $rc.|jjS)z A reference to the creating L{twisted.web.server.Site} object. Returns the site object from the backing channel. )rsiter7s rr z _GenericHTTPChannelProtocol.site s}}!!!rc4||_||j_y)a; A reference to the creating L{twisted.web.server.Site} object. Sets the object on the backing channel and also stores the value for propagation to any new channel. @param value: The L{twisted.web.server.Site} object to set. @type value: L{twisted.web.server.Site} N)_siterr rs rr z _GenericHTTPChannelProtocol.site s " rc.|jjS)z; The idle timeout for the backing channel. )rrr7s rrz#_GenericHTTPChannelProtocol.timeOut rrc4||_||j_y)a  The idle timeout for the backing channel. Sets the idle timeout on both the backing channel and stores it for propagation to any new backing channel. @param value: The timeout to set. @type value: L{int} or L{float} N)_timeOutrrrs rrz#_GenericHTTPChannelProtocol.timeOut$ s % rc.|jjS)z A value for the C{callLater} callback. This callback is used by the L{twisted.protocols.policies.TimeoutMixin} to handle timeouts. )rrr7s rrz%_GenericHTTPChannelProtocol.callLater2 s }}&&&rc4||_||j_y)z Sets the value for the C{callLater} callback. This callback is used by the L{twisted.protocols.policies.TimeoutMixin} to handle timeouts. @param value: The new callback to use. @type value: L{callable} N) _callLaterrrrs rrz%_GenericHTTPChannelProtocol.callLater: s "' rc^|jt |jjj}|d}|dk(r6t s t d|jj}|j|jjd|jj}t|_|j|j_ |j|j_|j|j_|j"|j_|j&|j_|jj+||j-|jdn |dk(sJd||_|jj/|S#t$rd}Y|wxYw)zi An override of L{IProtocol.dataReceived} that checks what protocol we're using. Nshttp/1.1sh2z"Negotiated HTTP/2 without support.TzUnsupported protocol negotiated)_negotiatedProtocolrrGnegotiatedProtocolAttributeError H2_ENABLEDrrrrrer rr r rrrrrrmakeConnectionrrA)r8rrnetworkProducerrGs rrAz(_GenericHTTPChannelProtocol.dataReceivedF sd  # # + 1%)]]%<%<%O%O" ")%0"!U*!$%IJJ #'--"@"@224 ((. MM33 , /3/C/C ,%)ZZ "(,  %(,  %*.// ' ,,Y7  00E'+55455(:D $}}))$//S" 1%0" 1s F F,+F,)rrrrrr=r rr rrrrsetterrr rrrArrrrr s @OH EHJ %%  ^^&&,, - -"" [[ # #%%  ^^ & &'' ( (10rrrc(ttS)zK Returns an appropriately initialized _GenericHTTPChannelProtocol. )rr?r7s r"_genericHTTPChannelProtocolFactoryrz s '{} 55rcPeZdZdZeZdZeZdeddfdZ dZ dZ dZ dZ dZd Zy) r@a Factory for HTTP server. @ivar _logDateTime: A cached datetime string for log messages, updated by C{_logDateTimeCall}. @type _logDateTime: C{str} @ivar _logDateTimeCall: A delayed call for the next update to the cached log datetime string. @type _logDateTimeCall: L{IDelayedCall} provided @ivar _logFormatter: See the C{logFormatter} parameter to L{__init__} @ivar _nativeize: A flag that indicates whether the log file being written to wants native strings (C{True}) or bytes (C{False}). This is only to support writing to L{twisted.python.log} which, unfortunately, works with native strings. @ivar reactor: An L{IReactorTime} provider used to manage connection timeouts and compute logging timestamps. Nc|sddlm}||_|tjj |}||_||_|t}||_d|_ d|_ y)a @param logPath: File path to which access log messages will be written or C{None} to disable logging. @type logPath: L{str} or L{bytes} @param timeout: The initial value of L{timeOut}, which defines the idle connection timeout in seconds, or C{None} to disable the idle timeout. @type timeout: L{float} @param logFormatter: An object to format requests into log lines for the access log. L{combinedLogFormatter} when C{None} is passed. @type logFormatter: L{IAccessLogFormatter} provider @param reactor: An L{IReactorTime} provider used to manage connection timeouts and compute logging timestamps. Defaults to the global reactor. r)reactorN) twisted.internetr osrabspathlogPathrr _logFormatter _logDateTime_logDateTimeCall)r8r$timeout logFormatterr s rr9zHTTPFactory.__init__ s\* 0  ggoog.G   /L)! $rct|jj|_|jj d|j |_y)zY Update log datetime periodically, so we aren't always recalculating it. rN)r4r rr&r_updateLogDateTimer'r7s rr+zHTTPFactory._updateLogDateTime s=0 0D0D0FG $ 6 6q$:Q:Q Rrctjj||}|jj|_|j |_|Sr6)rS ServerFactory buildProtocolr rr)r8addrps rr.zHTTPFactory.buildProtocol s?  " " 0 0t <ll,, LL rc|j|j|jr!|j|j|_yt j |_y)z6 Set up request logging if necessary. N)r'r+r$ _openLogFilelogFilerZlogfiler7s r startFactoryzHTTPFactory.startFactory sD  (  # # % <<,,T\\:DL;;DLrc t|dr9|jtjk7r|jj |`|j =|j j r"|j jd|_yyy)Nr3)rr3rZr4rr'activerr7s r stopFactoryzHTTPFactory.stopFactory sq 4 #||s{{* ""$  ,1F1F1M1M1O  ! ! ( ( *$(D !2P ,rc t|dd}|S)zP Override in subclasses, e.g. to use L{twisted.python.logfile}. abr)open)r8rfs rr2zHTTPFactory._openLogFile s tQ rc |j}|j|j|dz}|j|j dy#t $rYywxYw)z Write a line representing C{request} to the access log file. @param request: The request object about which to log. @type request: L{Request}  rN)r3r%r&r;r~r)r8rr3rs rrZzHTTPFactory.log sX /llG%%d&7&7ADHD MM$++f- .    s A AA)rrrrrrSr$_REQUEST_TIMEOUTrr9r+r.r5r8r2rZrrrr@r@ sJ42HGG$44QU#%JS  ') /rr@)rr r0r )rr rr r0zdict[bytes, list[bytes]])rrr6)rr r0r1)rr r0r)rr r0zTuple[bytes, bytes])r __future__r__all__rr!rrr"reryrremailrA email.messagerBiorCtypingrDrErFrGrHrI urllib.parserJrKrr1rzope.interfacerLrMrNrO incrementalrPr!rQrRrS"twisted.internet._producer_helpersrTtwisted.internet.deferrUtwisted.internet.interfacesrVtwisted.loggerrWtwisted.protocolsrXrYtwisted.pythonrZtwisted.python.compatr[r\twisted.python.componentsr]twisted.python.deprecater^twisted.python.failurer_twisted.web._responsesrr)rrrr&rrr+rr,r-r'r rr r r rrrr.rr(rrr rrr!rr"rr#r%r r/rr*rrrr$rtwisted.web.http_headersr`ratwisted.web.iwebrbrcrdtwisted.web._http2rer ImportErrorr?protocol_versionr0 responsesrrrrrr Exceptionrrr2r3r4r5r6r7rrr8r9rr: LineReceiverr;r<r/rr{compiler IConsumerr=r6r>r9r;rRrTrErrr ITransport TimeoutMixinr?rrIAddressrrrrrr-r@)rs0rrbs :#> B  $&@@ GF:::+1!-=7/*------------\HTT/J   @   /::dTZZ\:5>?T4(DJJL(?y &K8 8   ((49nE  -.$$\ \ ~ 1 1 Q###Q#j 8$# #CD Z ! !#LMNNNNb   GGT H8ppf Z % %&' ' '' T Z " "J$<$R>RSZ %$$h&;&;Z TZ z0 B Z !  " -(-h C-(` KK|0"3Iz"J|0~6F/(((F/K^JF;?s(M M"M MM