Ϫfe8ldZddlZddlmZmZmZmZmZmZm Z m Z ddl m Z ddl mZddlmZddlmZmZddlmZmZdd lmZdd lmZmZdd lmZdd lmZdd lm Z ddl!m"Z"ddl#m$Z$ddl%m&Z&ddl'm(Z(m)Z)m*Z*m+Z+ddl,m-Z-ddl.m/Z/ddl0m1Z1m2Z2ddl3m4Z4ddl5m6Z6m7Z7m8Z8edGdde9Z:GddeZ;GddeZ<e d Z=eeGd!d"e Z?Gd#d$e Z@eeGd%d&ZAeeGd'd(ZBGd)d*e9ZCGd+d,eZDy)-z: This module implements the worker classes. @since: 12.3 N)Any AwaitableCallableDictListOptionalTextIOTypeVar)TestCase) implementer)frozen)Protocol TypedDict)Deferred DeferredList) ProcessDone)IAddress ITransport)ProcessProtocol)Logger)AMP)Failure)FilePath) namedObject)_WORKER_AMP_STDIN_WORKER_AMP_STDOUTmanagercommandsworkercommands)WorkerReporter) TestResult) TestLoader TrialSuite)Todo) StreamOpenStreamReceiver StreamWriteF)auto_excceZdZUdZeed<y)WorkerExceptionz An exception was reported by a test running in a worker process. @ivar message: An error message describing the exception. messageN)__name__ __module__ __qualname____doc__str__annotations__s! h  y r3r:_TceZdZdZeZddZejjde de fdZ ejjdZy) WorkerProtocolz5 The worker-side trial distributed protocol. cPt|_t||_||_yN)r!_loaderr_result_forceGarbageCollection)r?forceGarbageCollections r4__init__zWorkerProtocol.__init__Ts!| %d+ '=$r3testCaser=c4K|jj5}|jj|}t |g|j }|j |jdddd}tdd{D]M\}}|r d}|jjd|| |jj||d{Od|iS#1swYtxYw7d7#t$r|jjdYwxYww) z* Run a test case by name. NT) consumeErrorsFz Result reporting for {id} failed)failureidz5Additionally, reporting the reporting failure failed.r7) rGgatherReportingResultsrF loadByNamer"rHr@rloggerrNaddErrorFallible BaseException)r?rKresultsr;suite allSucceededr7r<s r4r@zWorkerProtocol.runYs# \\ 0 0 2 $g<<**84Dvt'C'CDE IIdll # $  %1'%NN OGV!L KK  2   ll33' @<((M $ $ O& !  ##K  s_DAC+DC*-D4C.C,C.DC'#D,C..$DDDDc4tj|ddiS)zb Set up the worker, moving into given directory for tests to run in them. r7T)oschdir)r? directorys r4startzWorkerProtocol.starts 4  r3N)F)r,r-r.r/rrRrJrRun responderr0r6r@Startr\r2r3r4rCrCMsdXF> !!*)#*))*)"*)X##!$!r3rCc eZdZdZdfd Zej dZej dZ e jj dZ de dedeed efd Ze j$j d eded ed ed eeeff dZe j.j d ededed ed eeeff dZe j2j dZe j6j d ed edeed eeeffdZe j<j dZe j@j dZ!de"de#d e$fdZ%dZ&xZ'S)LocalWorkerAMPz7 Local implementation of the manager commands. cDt|||t|_yrE)superrJr&_streams)r? boxReceiverlocator __class__s r4rJzLocalWorkerAMP.__init__s g.&( r3c:d|jjiS)NstreamId)rdopenr?s r4 streamOpenzLocalWorkerAMP.streamOpensDMM..011r3c>|jj||iSrE)rdwrite)r?ridatas r4 streamWritezLocalWorkerAMP.streamWrites Hd+ r3cT|jj|jddiS)z0 Add a success to the reporter. r7T)rG addSuccess _testCase)r?testNames r4rrzLocalWorkerAMP.addSuccesss% /4  r3error errorClassframesr=c t|}t||}tdt|dD]:}|jj ||||dzt ||dzggf<|S)a Helper to build a C{Failure} with some traceback. @param error: An C{Exception} instance. @param errorClass: The class name of the C{error} class. @param frames: A flat list of strings representing the information need to approximatively rebuild C{Failure} frames. @return: A L{Failure} instance with enough information about a test error. rr$)rrrangelenrwappendint)r?rurvrw errorTyperNis r4 _buildFailurezLocalWorkerAMP._buildFailuresv& + %+q#f+q) A NN ! !F1q5M3va!e}+=r2F  r3rt errorStreamIdframesStreamIdcxdj|jj|jd}|jj|Dcgc]}|jd}}|j t |||}|j j|j|ddiScc}w)a Add an error to the reporter. @param errorStreamId: The identifier of a stream over which the text of this error was previously completely sent to the peer. @param framesStreamId: The identifier of a stream over which the lines of the traceback for this error were previously completely sent to the peer. @param error: A message describing the error. r3utf-8r7T) joinrdfinishdecoderr*rGaddErrorrs) r?rtrvrrruframerwrNs r4rzLocalWorkerAMP.addErrors(--m<=DDWM/3}}/C/CN/S &+ELL !  $$_U%;ZP dnng64   B7 failStreamId failClasscxdj|jj|jd}|jj|Dcgc]}|jd}}|j t |||}|j j|j|ddiScc}w)az Add a failure to the reporter. @param failStreamId: The identifier of a stream over which the text of this failure was previously completely sent to the peer. @param framesStreamId: The identifier of a stream over which the lines of the traceback for this error were previously completely sent to the peer. r3rr7T) rrdrrrr*rG addFailurers) r?rtrrrfailrrwrNs r4rzLocalWorkerAMP.addFailures$xx ,,\:;BB7K/3}}/C/CN/S &+ELL !  $$_T%:IvN 84   rcV|jj|j|ddiS)z- Add a skip to the reporter. r7T)rGaddSkiprs)r?rtreasons r4rzLocalWorkerAMP.addSkips' T^^V44  r3todocdj|jj|jd}t |dn|}|j j |j||ddiS)z Add an expected failure to the reporter. @param errorStreamId: The identifier of a stream over which the text of this error was previously completely sent to the peer. r3rz r7T)rrdrrr#rGaddExpectedFailurers)r?rtrrru_todos r4rz!LocalWorkerAMP.addExpectedFailuresa--m<=DDWMDL[d; ''uE4  r3cV|jj|j|ddiS)z< Add an unexpected success to the reporter. r7T)rGaddUnexpectedSuccessrs)r?rtrs r4rz#LocalWorkerAMP.addUnexpectedSuccesss' ))$..$?4  r3cz|jj|dz|jjddiS)z4 Print test output from the worker.  r7T) _testStreamrnflush)r?outs r4 testWritezLocalWorkerAMP.testWrites9 sTz*  4  r3rKr<cJK||_||_|jj||j} |j t j |d{|jj|S7#|jj|wxYww)z Run a test. )rKN)rsrG startTestrO callRemoterr]stopTest)r?rKr< testCaseIds r4r@zLocalWorkerAMP.run(s|"  x([[]  ,););jQQ LL ! !( +R LL ! !( +s.:B#$B!B"B%B#BB  B#c||_y)z? Set the stream used to log output from tests. N)r)r?streams r4 setTestStreamzLocalWorkerAMP.setTestStream5s "r3)NN)(r,r-r.r/rJr%r^rlr'rpr AddSuccessrrr*r0rrrAddErrorr~rr8r AddFailurerAddSkiprAddExpectedFailurerrAddUnexpectedSuccessr TestWriterr r r6r@r __classcell__)rgs@r4raras )22))!*!S   6''!!! !  ! c4i !(!>))!!! !  ! c4i !*!4&&!'!''11 ! !,/ !7?} ! c4i !2 !))33!4!((!)! ,( ,J ,9 ,"r3raceZdZdZy)LocalWorkerAddressz A L{IAddress} implementation meant to provide stub addresses for L{ITransport.getPeer} and L{ITransport.getHost}. Nr,r-r.r/r2r3r4rr<sr3rc4eZdZdZdZdZdZdZdZdZ y) LocalWorkerTransportzi A stub transport implementation used to support L{AMP} over a L{ProcessProtocol} transport. c||_yrE) _transport)r? transports r4rJzLocalWorkerTransport.__init__Ks #r3cD|jjt|y)z, Forward data to transport. Nr writeToChildrr?ros r4rnzLocalWorkerTransport.writeNs $$%6=r3cR|D]"}|jjt|$y)zP Emulate C{writeSequence} by iterating data in the C{sequence}. Nr)r?sequenceros r4 writeSequencez"LocalWorkerTransport.writeSequenceTs* BD OO ( ():D A Br3c8|jjy)z' Closes the transport. N)rloseConnectionrks r4rz#LocalWorkerTransport.loseConnection[s &&(r3ctSz: Return a L{LocalWorkerAddress} instance. rrks r4getHostzLocalWorkerTransport.getHosta "##r3ctSrrrks r4getPeerzLocalWorkerTransport.getPeergrr3N) r,r-r.r/rJrnrrrrr2r3r4rrDs& $> B) $ $r3rceZdZdZy) NotRunningzN An operation was attempted on a worker process which is not running. Nrr2r3r4rrnsr3rc`eZdZdZdedeedefdZddZ d Z d Z d e ddfd Z d ZdZdZy) LocalWorkeraL Local process worker protocol. This worker runs as a local process and communicates via stdin/out. @ivar _ampProtocol: The L{AMP} protocol instance used to communicate with the worker. @ivar _logDirectory: The directory where logs will reside. @ivar _logFile: The main log file for tests output. ampProtocol logDirectorylogFilecL||_||_||_t|_yrE) _ampProtocol _logDirectory_logFiler endDeferred)r?rrrs r4rJzLocalWorker.__init__s% () +3:r3r=NcK|j t|j}|jjt |d{y7#t $rYywxYww)z3 Cause the worker process to exit. N)rrr closeChildFDrr)r?rs r4exitzLocalWorker.exitsW >> !, &&  ##$56      s<AA!A A AA!A AA!AA!c@|jjt|j|jj d|jj djd|_|jj djd|_ |jj|j|jjtj|jj}|j!dy) zL When connection is made, create the AMP protocol instance. T)ignoreExistingDirectoryzout.logwzerr.log)r[cyrEr2)xs r4z,LocalWorker.connectionMade..sr3N)rmakeConnectionrrrmakedirschildrj_outLog_errLogrrrrr_path addErrback)r?ds r4connectionMadezLocalWorker.connectionMades (()=dnn)MN ##D#A))// :??D ))// :??D  '' 6    ( (  ((-- )  ^$r3cz|jj|jjd|_y)zk On connection lost, close the log files that we're managing for stdin and stdout. N)rcloserrr?rs r4connectionLostzLocalWorker.connectionLosts+  r3rc|j||jj||jj|y)z When the process closes, call C{connectionLost} for cleanup purposes and forward the information to the C{_ampProtocol}. N)rrrcallbackrs r4 processEndedzLocalWorker.processEndeds: F# ((0 !!&)r3c:|jj|y)z8 Send data received from stdout to log. N)rrnrs r4 outReceivedzLocalWorker.outReceiveds 4 r3c:|jj|y)z* Write error data to log. N)rrnrs r4 errReceivedzLocalWorker.errReceiveds 4 r3c||tk(r|jj|ytj|||y)zT Handle data received on the specific pipe for the C{_ampProtocol}. N)rr dataReceivedrchildDataReceived)r?childFDros r4rzLocalWorker.childDataReceiveds3 ( (    * *4 0  - -dGT Br3)r=N)r,r-r.r/rarrr rJrrrrrrrrr2r3r4rrts^  6# 6sm 6 6 %"*7*t*!! Cr3r)Er/rYtypingrrrrrrr r unittestr zope.interfacer attrsr rrtwisted.internet.deferrrtwisted.internet.errorrtwisted.internet.interfacesrrtwisted.internet.protocolrtwisted.loggerrtwisted.protocols.amprtwisted.python.failurertwisted.python.filepathrtwisted.python.reflectrtwisted.trial._distrrrr"twisted.trial._dist.workerreporterrtwisted.trial.reporterr twisted.trial.runnerr!r"twisted.trial.unittestr#rr%r&r' Exceptionr*r6r:rA WorkerActionrCrarrrrr2r3r4r s,   RRR&&9.<5!%*,. >-7';;i  X  T])B-/0 @!S@!Fi"Si"X X Z&$&$&$R ]C/]Cr3