ϪfdZddlmZddlmZmZmZmZddlm Z m Z m Z ddl m Z mZmZmZmZddlmZddlmZddlmZmZmZmZmZmZmZmZmZm Z m!Z!m"Z"dd l#m$Z$dd l%m&Z&dd l'm(Z(m)Z)dd l*m+Z+e eGd dZ,ddZ-e e!ee"Gdde(Z.e eeGddZ/GddZ0ddZ1Gdde.Z2Gdde)Z3y)a Implementation of a TLS transport (L{ISSLTransport}) as an L{IProtocol} layered on top of any L{ITransport} implementation, based on U{OpenSSL}'s memory BIO features. L{TLSMemoryBIOFactory} is a L{WrappingFactory} which wraps protocols created by the factory it wraps with L{TLSMemoryBIOProtocol}. L{TLSMemoryBIOProtocol} intercedes between the underlying transport and the wrapped protocol to implement SSL and TLS. Typical usage of this module looks like this:: from twisted.protocols.tls import TLSMemoryBIOFactory from twisted.internet.protocol import ServerFactory from twisted.internet.ssl import PrivateCertificate from twisted.internet import reactor from someapplication import ApplicationProtocol serverFactory = ServerFactory() serverFactory.protocol = ApplicationProtocol certificate = PrivateCertificate.loadPEM(certPEMData) contextFactory = certificate.options() tlsFactory = TLSMemoryBIOFactory(contextFactory, False, serverFactory) reactor.listenTCP(12345, tlsFactory) reactor.run() This API offers somewhat more flexibility than L{twisted.internet.interfaces.IReactorSSL}; for example, a L{TLSMemoryBIOProtocol} instance can use another instance of L{TLSMemoryBIOProtocol} as its transport, yielding TLS over TLS - useful to implement onion routing. It can also be used to run TLS over unusual transports, such as UNIX sockets and stdio. ) annotations)CallableIterableOptionalcast)directlyProvides implementer providedBy) ConnectionError SysCallError WantReadErrorZeroReturnError) _PullToPush)_setAcceptableProtocols) IDelayedCallIHandshakeListenerILoggingContext INegotiatedIOpenSSLClientConnectionCreatorIOpenSSLServerConnectionCreator IProtocolIProtocolNegotiationFactory IPushProducer IReactorTime ISystemHandle ITransport)CONNECTION_LOST)Protocol)ProtocolWrapperWrappingFactory)Failurec,eZdZdZdZdZdZdZdZy)_ProducerMembranea Stand-in for producer registered with a L{TLSMemoryBIOProtocol} transport. Ensures that producer pause/resume events from the undelying transport are coordinated with pause/resume events from the TLS layer. @ivar _producer: The application-layer producer. Fc||_yN) _producer)selfproducers 7/usr/lib/python3/dist-packages/twisted/protocols/tls.py__init__z_ProducerMembrane.__init__Rs !c`|jryd|_|jjy)zP C{pauseProducing} the underlying producer, if it's not paused. NT)_producerPausedr'pauseProducingr(s r*r/z _ProducerMembrane.pauseProducingUs(    # %%'r,c`|jsyd|_|jjy)zM C{resumeProducing} the underlying producer, if it's paused. NF)r.r'resumeProducingr0s r*r2z!_ProducerMembrane.resumeProducing^s(## $ &&(r,c8|jjy)z C{stopProducing} the underlying producer. There is only a single source for this event, so it's simply passed on. N)r' stopProducingr0s r*r4z_ProducerMembrane.stopProducinggs $$&r,N) __name__ __module__ __qualname____doc__r.r+r/r2r4r,r*r$r$Es"O"()'r,r$ct|tr|j\}}n|jd}|d\}}}|jj dS)zD Does the given OpenSSL.SSL.Error represent an end-of-file? rzunexpected eof) isinstancer argscasefold startswith)exceptionObject_ reasonString errorQueues r*_representsEOFrDqsX /<0)..<$))!, '^1l  " - -.> ??r,ceZdZdZdZdZdZdZdZddZ dZ dZ dZ dZ d Zd Zd Zd Zd ZdZdZdZdZdZdZdZdZdZedZdZdZy)TLSMemoryBIOProtocola L{TLSMemoryBIOProtocol} is a protocol wrapper which uses OpenSSL via a memory BIO to encrypt bytes written to it before sending them on to the underlying transport and decrypts bytes received from the underlying transport before delivering them to the wrapped protocol. In addition to producer events from the underlying transport, the need to wait for reads before a write can proceed means the L{TLSMemoryBIOProtocol} may also want to pause a producer. Pause/resume events are therefore merged using the L{_ProducerMembrane} wrapper. Non-streaming (pull) producers are supported by wrapping them with L{_PullToPush}. Because TLS may need to wait for reads before writing, some writes may be buffered until a read occurs. @ivar _tlsConnection: The L{OpenSSL.SSL.Connection} instance which is encrypted and decrypting this connection. @ivar _lostTLSConnection: A flag indicating whether connection loss has already been dealt with (C{True}) or not (C{False}). TLS disconnection is distinct from the underlying connection being lost. @ivar _appSendBuffer: application-level (cleartext) data that is waiting to be transferred to the TLS buffer, but can't be because the TLS connection is handshaking. @type _appSendBuffer: L{list} of L{bytes} @ivar _connectWrapped: A flag indicating whether or not to call C{makeConnection} on the wrapped protocol. This is for the reactor's L{twisted.internet.interfaces.ITLSTransport.startTLS} implementation, since it has a protocol which it has already called C{makeConnection} on, and which has no interest in a new transport. See #3821. @ivar _handshakeDone: A flag indicating whether or not the handshake is known to have completed successfully (C{True}) or not (C{False}). This is used to control error reporting behavior. If the handshake has not completed, the underlying L{OpenSSL.SSL.Error} will be passed to the application's C{connectionLost} method. If it has completed, any unexpected L{OpenSSL.SSL.Error} will be turned into a L{ConnectionLost}. This is weird; however, it is simply an attempt at a faithful re-implementation of the behavior provided by L{twisted.internet.ssl}. @ivar _reason: If an unexpected L{OpenSSL.SSL.Error} occurs which causes the connection to be lost, it is saved here. If appropriate, this may be used as the reason passed to the application protocol's C{connectionLost} method. @ivar _producer: The current producer registered via C{registerProducer}, or L{None} if no producer has been registered or a previous one was unregistered. @ivar _aborted: C{abortConnection} has been called. No further data will be received to the wrapped protocol's C{dataReceived}. @type _aborted: L{bool} NFc@tj|||||_yr&)r r+_connectWrapped)r(factorywrappedProtocolrHs r*r+zTLSMemoryBIOProtocol.__init__s  w@.r,c|jS)ai Return the L{OpenSSL.SSL.Connection} object being used to encrypt and decrypt this connection. This is done for the benefit of L{twisted.internet.ssl.Certificate}'s C{peerFromTransport} and C{hostFromTransport} methods only. A different system handle may be returned by future versions of this method. )_tlsConnectionr0s r* getHandlezTLSMemoryBIOProtocol.getHandles"""r,cP|jj||_g|_t |D]}t ||t j|||jj||jrtj|||jy)z Connect this wrapper to the given transport and initialize the necessary L{OpenSSL.SSL.Connection} with a memory BIO. N) rI_createConnectionrL_appSendBufferr rrmakeConnectionregisterProtocolrHr _checkHandshakeStatus)r( transport interfaces r*rQz#TLSMemoryBIOProtocol.makeConnections #ll<++007(}}#00u=/))8 3! " 0!!#))$/ 3 ")))'2  3sA** C5)C #CCc|jj||js|j|jsy|jr|j |j y)z Deliver any received bytes to the receive BIO and then read and deliver to the application any application-level data which becomes available as a result of this. N)rL bio_writerYrSrP_unbufferPendingWritesriras r*rfz!TLSMemoryBIOProtocol.dataReceived8sb %%e,""  & & (&&     ' ' ) r,c |jj}|j|r|jj yy#t$rd}Y;wxYw)zQ Initiate, or reply to, the shutdown handshake of the TLS layer. FN)rLshutdownr r[rTloseConnection)r(shutdownSuccesss r*rgz!TLSMemoryBIOProtocol._shutdownTLSUs[ $"11::+>  "!""t~~'=    (>"r,ctd|_d|_|j|jj y)z Tear down TLS state so that if the connection is aborted mid-handshake we don't deliver any further data from the application. TN)rWr{rgrTr|r0s r*r|z$TLSMemoryBIOProtocol.abortConnections/  !  &&(r,c2||_|jy)a, Abort the connection during connection setup, giving a reason that certificate verification failed. @param reason: The reason that the verification failed; reported to the application protocol's C{connectionLost} method. @type reason: L{Failure} N)rsr|rts r*failVerificationz%TLSMemoryBIOProtocol.failVerifications  r,cX|jr |jy|j|y)z Process the given application bytes and send any resulting TLS traffic which arrives in the send BIO. If C{loseConnection} was called, subsequent calls to C{write} will drop the bytes on the floor. N)r{r'_writeras r*r`zTLSMemoryBIOProtocol.writes%   $.."8  Er,c|jj||j|jjyy)z Put the given octets into L{TLSMemoryBIOProtocol._appSendBuffer}, and tell any listening producer that it should pause because we are now buffering. N)rPappendr'r/)r(octetss r*_bufferedWritez#TLSMemoryBIOProtocol._bufferedWrites7 ""6* >> % NN ) ) + &r,c|jgc}|_|D]}|j||jry|j|jjy|jr|j yy)zY Un-buffer all waiting writes in L{TLSMemoryBIOProtocol._appSendBuffer}. N)rPrr'r2r{rg)r( pendingWrites eachWrites r*rlz+TLSMemoryBIOProtocol._unbufferPendingWritess.2-@-@"* t*& #I KK " #     >> % NN * * ,         r,c`|jryd}d}|t|krI||||z} |jj|}||z }|j |t|krHyy#t $r|j ||dYyt$r|jtYywxYw)a^ Process the given application bytes and send any resulting TLS traffic which arrives in the send BIO. This may be called by C{dataReceived} with bytes that were buffered before C{loseConnection} was called, which is why this function doesn't check for disconnection but accepts the bytes regardless. Ni@r) rdlenrLsendr[rrr r\r")r(rb bufferSize alreadySenttoSendsents r*rzTLSMemoryBIOProtocol._writes  " "   CJ&;z)ABF %**//7t# ""$+CJ&! ##E+,$78  ))')4 sA**B- !B-,B-cD|jdj|y)z} Write a sequence of application bytes by joining them into one string and passing them to L{write}. r,N)r`join)r(iovecs r* writeSequencez"TLSMemoryBIOProtocol.writeSequence"s 388E?#r,c6|jjSr&)rLget_peer_certificater0s r*getPeerCertificatez'TLSMemoryBIOProtocol.getPeerCertificate)s""7799r,cd} |jj}|dvr|S |jj }|dk7r|Sy#ttf$rY:wxYw#ttf$rY.wxYw)z9 @see: L{INegotiated.negotiatedProtocol} N)r,Nr,)rLget_alpn_proto_negotiatedNotImplementedErrorAttributeErrorget_next_proto_negotiated)r( protocolNames r*negotiatedProtocolz'TLSMemoryBIOProtocol.negotiatedProtocol,s   ..HHJL { *  ..HHJL 3  $^4   $^4   s"AAAAA.-A.c|jr|jy|st||x}}t|}|jj |d||_|sjyy)NT)rdr4rr$rTregisterProducerr'startStreaming)r(r) streamingstreamingProducers r*rz%TLSMemoryBIOProtocol.registerProducerGsl  " "  " " $ +6x+F FH($X. ''$7!  , , .r,cR|jyt|jjtr$|jjjd|_d|_|j j |jr|js|jyyy)NF) r'r<r stopStreamingr.rTunregisterProducerr{rPrgr0s r*rz'TLSMemoryBIOProtocol.unregisterProducerZs >> !  dnn.. < NN $ $ 2 2 4$ ))+   d&9&9    ': r,T) r5r6r7r8rsrYrdr'rWr+rMrQrSr[rirfrgr\ryror|rr`rrlrrrpropertyrrrr9r,r*rFrF~s7rGNIH/ #%4:0 ((T :,((B#( $)  , 4'%R$:4/&  r,rFc(eZdZdZdZdZdZdZy)"_ContextFactoryToConnectionFactorya Adapter wrapping a L{twisted.internet.interfaces.IOpenSSLContextFactory} into a L{IOpenSSLClientConnectionCreator} or L{IOpenSSLServerConnectionCreator}. See U{https://twistedmatrix.com/trac/ticket/7215} for work that should make this unnecessary. c2|j||_y)a( Construct a L{_ContextFactoryToConnectionFactory} with a L{twisted.internet.interfaces.IOpenSSLContextFactory}. Immediately call C{getContext} on C{oldStyleContextFactory} in order to force advance parameter checking, since old-style context factories don't actually check that their arguments to L{OpenSSL} are correct. @param oldStyleContextFactory: A factory that can produce contexts. @type oldStyleContextFactory: L{twisted.internet.interfaces.IOpenSSLContextFactory} N) getContext_oldStyleContextFactory)r(oldStyleContextFactorys r*r+z+_ContextFactoryToConnectionFactory.__init__us ))+'=$r,cN|jj}t|dS)z Create an L{OpenSSL.SSL.Connection} object. @param protocol: The protocol initiating a TLS connection. @type protocol: L{TLSMemoryBIOProtocol} @return: a connection @rtype: L{OpenSSL.SSL.Connection} N)rrr )r(protocolcontexts r*_connectionForTLSz4_ContextFactoryToConnectionFactory._connectionForTLSs%..99;'4((r,c$|j|S)a Construct an OpenSSL server connection from the wrapped old-style context factory. @note: Since old-style context factories don't distinguish between clients and servers, this is exactly the same as L{_ContextFactoryToConnectionFactory.clientConnectionForTLS}. @param protocol: The protocol initiating a TLS connection. @type protocol: L{TLSMemoryBIOProtocol} @return: a connection @rtype: L{OpenSSL.SSL.Connection} rr(rs r*serverConnectionForTLSz9_ContextFactoryToConnectionFactory.serverConnectionForTLS%%h//r,c$|j|S)a Construct an OpenSSL server connection from the wrapped old-style context factory. @note: Since old-style context factories don't distinguish between clients and servers, this is exactly the same as L{_ContextFactoryToConnectionFactory.serverConnectionForTLS}. @param protocol: The protocol initiating a TLS connection. @type protocol: L{TLSMemoryBIOProtocol} @return: a connection @rtype: L{OpenSSL.SSL.Connection} rrs r*clientConnectionForTLSz9_ContextFactoryToConnectionFactory.clientConnectionForTLSrr,N)r5r6r7r8r+rrrr9r,r*rrjs> )0"0r,rc4eZdZdZdZddZd dZd dZd dZy) _AggregateSmallWritesam Aggregate small writes so they get written in large batches. If this is used as part of a transport, the transport needs to call ``flush()`` immediately when ``loseConnection()`` is called, otherwise any buffered writes will never get written. @cvar MAX_BUFFER_SIZE: The maximum amount of bytes to buffer before writing them out. ic^||_||_g|_|j|_d|_yr&)r_clock_bufferMAX_BUFFER_SIZE _bufferLeft _scheduled)r(r`clocks r*r+z_AggregateSmallWrites.__init__s,  $& //26r,c&|jj||xjt|zc_|jdkr|j y|j ry|j jd|j|_y)z Buffer the data, or write it immediately if we've accumulated enough to make it worth it. Accumulating too much data can result in higher memory usage. rN) rrrrflushrr callLater_scheduledFlush)r(datas r*r`z_AggregateSmallWrites.writess D! CI%   a  JJL  ?? ++//43G3GHr,c2d|_|jy)z!Called in next reactor iteration.N)rrr0s r*rz%_AggregateSmallWrites._scheduledFlushs r,c|jrK|j|_|jdj |j|jdd=yy)zFlush any buffered writes.r,N)rrrrrr0s r*rz_AggregateSmallWrites.flushs@ <<#33D  KK. / Q r,N)r`zCallable[[bytes], object]rr)rrbreturnNonerr) r5r6r7r8rr+r`rrr9r,r*rrs# O7I2  r,rc.ddlm}tt|S)z Return the default reactor. This is a function so it can be monkey-patched in tests, specifically L{twisted.web.test.test_agent}. rreactor)twisted.internetrrrrs r*_get_default_clockrs)  g &&r,cFeZdZdZ d dfd ZddZdfd ZxZS) BufferingTLSTransporta A TLS transport implemented by wrapping buffering around a L{TLSMemoryBIOProtocol}. Doing many small writes directly to a L{OpenSSL.SSL.Connection}, as implemented in L{TLSMemoryBIOProtocol}, can add significant CPU and bandwidth overhead. Thus, even when writing is possible, small writes will get aggregated and written as a single write at the next reactor iteration. ct||||t|}t||j|_|j j|_yr&)superr+r`rr _aggregator)r(rIrJrH actual_write __class__s r*r+zBufferingTLSTransport.__init__sG /?Cw} 0w~~N%%++ r,cX|jjdj|y)Nr,)rr`r)r(sequences r*rz#BufferingTLSTransport.writeSequences sxx12r,cV|jjt| yr&)rrrro)r(rs r*roz$BufferingTLSTransport.loseConnections     r,r)rITLSMemoryBIOFactoryrJrrHbool)rzIterable[bytes]rrr)r5r6r7r8r+rro __classcell__)rs@r*rrs=*!% ,$ ,# , ,3!!r,rc4eZdZdZeZdZ ddZdZdZ dZ y) ra L{TLSMemoryBIOFactory} adds TLS to connections. @ivar _creatorInterface: the interface which L{_connectionCreator} is expected to implement. @type _creatorInterface: L{zope.interface.interfaces.IInterface} @ivar _connectionCreator: a callable which creates an OpenSSL Connection object. @type _connectionCreator: 1-argument callable taking L{TLSMemoryBIOProtocol} and returning L{OpenSSL.SSL.Connection}. FNctj|||rt}nt}||_|j |s t |}||_| t}||_ y)a2 Create a L{TLSMemoryBIOFactory}. @param contextFactory: Configuration parameters used to create an OpenSSL connection. In order of preference, what you should pass here should be: 1. L{twisted.internet.ssl.CertificateOptions} (if you're writing a server) or the result of L{twisted.internet.ssl.optionsForClientTLS} (if you're writing a client). If you want security you should really use one of these. 2. If you really want to implement something yourself, supply a provider of L{IOpenSSLClientConnectionCreator} or L{IOpenSSLServerConnectionCreator}. 3. If you really have to, supply a L{twisted.internet.ssl.ContextFactory}. This will likely be deprecated at some point so please upgrade to the new interfaces. @type contextFactory: L{IOpenSSLClientConnectionCreator} or L{IOpenSSLServerConnectionCreator}, or, for compatibility with older code, anything implementing L{twisted.internet.interfaces.IOpenSSLContextFactory}. See U{https://twistedmatrix.com/trac/ticket/7215} for information on the upcoming deprecation of passing a L{twisted.internet.ssl.ContextFactory} here. @param isClient: Is this a factory for TLS client connections; in other words, those that will send a C{ClientHello} greeting? L{True} if so, L{False} otherwise. This flag determines what interface is expected of C{contextFactory}. If L{True}, C{contextFactory} should provide L{IOpenSSLClientConnectionCreator}; otherwise it should provide L{IOpenSSLServerConnectionCreator}. @type isClient: L{bool} @param wrappedFactory: A factory which will create the application-level protocol. @type wrappedFactory: L{twisted.internet.interfaces.IProtocolFactory} N) r!r+rr_creatorInterfacer r_connectionCreatorrr)r(contextFactoryisClientwrappedFactoryrcreatorInterfaces r*r+zTLSMemoryBIOFactory.__init__6s`b   ~6 > > !1**>:?ON"0 =&(E r,ctj|jr|jj}n |jjj }|dS)z Annotate the wrapped factory's log prefix with some text indicating TLS is in use. @rtype: C{str} z (TLS))rr r logPrefixrr5)r(rs r*rzTLSMemoryBIOFactory.logPrefixusP  % %d&9&9 :++557I++55>>IF##r,ctj|jr6|jj}|j }t ||y)a8 Applies ALPN/NPN protocol neogitation to the connection, if the factory supports it. @param connection: The OpenSSL connection object to have ALPN/NPN added to it. @type connection: L{OpenSSL.SSL.Connection} @return: Nothing @rtype: L{None} N)rr racceptableProtocols get_contextr)r( connection protocolsrs r*_applyProtocolNegotiationz-TLSMemoryBIOFactory._applyProtocolNegotiationsG ' 1 1$2E2E F++??AI ,,.G #GY 7r,c|j}|jtur4|j|}|j ||j |S|j |}|j ||j|S)a8 Create an OpenSSL connection and set it up good. @param tlsProtocol: The protocol which is establishing the connection. @type tlsProtocol: L{TLSMemoryBIOProtocol} @return: an OpenSSL connection object for C{tlsProtocol} to use @rtype: L{OpenSSL.SSL.Connection} )rrrrrset_connect_staterset_accept_state)r( tlsProtocolconnectionCreatorrs r*rOz%TLSMemoryBIOFactory._createConnections!33  ! !%D D*AA+NJ  * *: 6  ( ( * +AA+NJ  * *: 6  ' ' )r,r&) r5r6r7r8rrnoisyr+rrrOr9r,r*rr$s- %H E =~ $&r,rN)r@r rr)rr)4r8 __future__rtypingrrrrzope.interfacerr r OpenSSL.SSLr r r rr"twisted.internet._producer_helpersrtwisted.internet._sslverifyrtwisted.internet.interfacesrrrrrrrrrrrrtwisted.internet.mainrtwisted.internet.protocolrtwisted.protocols.policiesr r!twisted.python.failurer"r$rDrFrrrrrr9r,r*rs D#55DDWW:?    2.G* ]('('('V @ ]K4h ?h 5h V ,.MNG0G0OG0T8 8 v '%!0%!PD/Dr,