\jxdZddlZddlZddlZddlZddlmZeje Z dZ dZ dZ dZdZGd d ej ZGd d ej Zy) z,Implements reading and writing to/from HDFS.N)utils)hdfsviewfs)zhdfs:///path/filezhdfs://path/filezviewfs:///path/filezviewfs://path/filectjj|}|jtvsJ|j |j z}d|jdz}|std|zt|j|S)N/zinvalid HDFS URI: %r)schemeuri_path) urllibparseurlsplitrSCHEMESnetlocpathlstrip RuntimeErrordict) uri_as_string split_urir s 9/root/env/lib/python3.12/site-packages/smart_open/hdfs.py parse_urirsx %%m4I   w && &)..0HX__S))H 1MABB y''( ;;ctjt|t|}t|d|}|dj dd|_|S)Nr r)r check_kwargsopenrsplitname)urimodetransport_params parsed_urifobjs ropen_urir#*sN t-.3J  :& -D:&,,S1"5DI Krc^|dk(r t|S|dk(r t|Std|z)Nrbwbz(hdfs support for mode %r not implemented)CliRawInputBaseCliRawOutputBaseNotImplementedError)rrs rrr3s7 t|s## $$!"Lt"STTrcXeZdZdZdZdZdZedZdZ dZ dZ d d Z d d Z d Zy) r'zReads bytes from HDFS via the "hdfs dfs" command-line interface. Implements the io.RawIOBase interface of the standard library. Nc||_tjddd|jgtj|_y)Nrdfsz-cat)stdout_uri subprocessPopenPIPE_subselfrs r__init__zCliRawInputBase.__init__Cs1 $$feVTYY%GPZP_P_` rctjd|js"|jj d|_yy)zFlush and close this stream. close: calledN)loggerdebugclosedr3 terminater5s rclosezCliRawInputBase.closeJs1 _%{{ II   !DIrc|jduSNr3r=s rr;zCliRawInputBase.closedQyyD  rc|jduS)z+Return True if the stream can be read from.NrAr=s rreadablezCliRawInputBase.readableUyy$$rcyz;If False, seek(), tell() and truncate() will raise IOError.Fr=s rseekablezCliRawInputBase.seekableYrc"tj)z Unsupported.ioUnsupportedOperationr=s rdetachzCliRawInputBase.detach`s%%%rcL|jjj|S)z6Read up to size bytes from the object and return them.)r3r-readr5sizes rrQzCliRawInputBase.readdsyy$$T**rc&|j|S)zThis is the same as read().)rS)rQrRs rread1zCliRawInputBase.read1hsyydy##rcn|jt|}|sy||dt|t|S)zLRead up to len(b) bytes into b, and return the number of bytes read.rN)rQlen)r5bdatas rreadintozCliRawInputBase.readintols5yyQ *3t9 4yr)r)__name__ __module__ __qualname____doc__r3r6r>propertyr;rDrIrOrQrUrZrHrrr'r'<sK Da!!%&+$rr'cNeZdZdZdZdZdZedZdZ dZ dZ d Z d Z y) r(zWrites bytes to HDFS via the "hdfs dfs" command-line interface. Implements the io.RawIOBase interface of the standard library. Nc||_tjddddd|jgtj|_y)Nrr,z-putz-f-)stdinr.r4s rr6zCliRawOutputBase.__init__}s5 $$feVT3 %R+5??< rctjd|jsV|j|jj j |jjd|_yy)Nr8)r9r:r;flushr3rcr>waitr=s rr>zCliRawOutputBase.closesL _%{{ JJL IIOO ! ! # IINN DI rc|jduSr@rAr=s rr;zCliRawOutputBase.closedrBrcL|jjjyr@)r3rcrer=s rrezCliRawOutputBase.flushs rc|jduS)z(Return True if this object is writeable.NrAr=s r writeablezCliRawOutputBase.writeablerErcyrGrHr=s rrIzCliRawOutputBase.seekablerJrcL|jjj|S)aWrite the given buffer to the underlying raw stream. Returns the number of bytes written, as required by :class:`io.RawIOBase`. Without this return value, callers that wrap this stream and rely on the documented ``write`` contract (for example, ``ray._private.external_storage._write_multiple_objects``, which asserts ``written_bytes == payload_len``) fail with an ``AssertionError`` because ``write`` would otherwise implicitly return ``None``. )r3rcwrite)r5rXs rrmzCliRawOutputBase.writesyy$$Q''rc,tjd)Nzdetach() not supportedrLr=s rrOzCliRawOutputBase.detachs%%&>??r)r[r\r]r^r3r6r>r_r;rerjrIrmrOrHrrr(r(vsF D< !! % ( @rr()r^rMloggingr0 urllib.parser smart_openr getLoggerr[r9r URI_EXAMPLESrr#r RawIOBaser'r(rHrrrusj3    8 $   <U7bll7t4@r||4@r