]j{9pddlZddlZddlZddlZddlmZddlmZddlm Z ddl m Z ddl m Z mZmZmZddlmZddlmZdd lmZdd lmZdd lmZdd lmZdd lmZddlm Z ddl!m"Z"ejFe$Z%GddZ&edGdde&eZ'edGdde&eZ(y)N)Union)unquote)extract_message)settings) HttpRequest HttpResponseHttpResponseRedirectStreamingHttpResponse) extend_schema)Projectstatus)IsAuthenticated)Response)APIView)Task)get_storage_by_url) parse_rangecheZdZdededeeefdefdZ dedeeefdedefdZ dZ d Z d Z d Zy ) ResolveStorageUriAPIMixinrequestfileuriinstancereturnc Rt|j}|j|jst t j S tj|jj}t!|t"r|n |j$}|j&}t)||}|s2tj+d|t t j,St/|ds3tj+d |d t t j,S|j0} | r|j3|||S|j5||||S#t$rB}tjd|d|d|jd|d t|}Yd}~$d}~wwxYw) Nr zFailed to decode base64  for  : z falling back to unquotezCould not find storage for URI presignzStorage z does not support presign URLs)type__name__has_permissionuserrrHTTP_403_FORBIDDENbase64urlsafe_b64decodeencodedecode Exceptionloggerdebugidr isinstancer projectget_all_import_storage_objectsrerrorHTTP_404_NOT_FOUNDhasattrrredirect_to_presign_urlproxy_data_from_storage) selfrrr model_nameexcr.storage_objectsstoragers L/root/env/lib/python3.12/site-packages/label_studio/io_storages/proxy_api.pyresolvez!ResolveStorageUriAPIMixin.resolvesk(^,, &&w||46#<#<= = '..w~~/?@GGIG)7;(AQAQ!@@$Wo> LL:7)D E6#<#<= =w * LL8G9,JK L6#<#<= =// //:N N//'7S S9 ' LL*7)5 Ahkk]RTUXTYYqr g&G  's 1E F&$7F!!F&r6c \ |j|}||jdt t jS|d}d}|jd r|jd d z}t|t j }d ||jd <|jjdd|jjdd|S#t$rO}tjd|d|d|jd|t t jcYd}~Sd}~wwxYw)z?Generate and redirect to a presigned URL for the given file URIzFailed to resolve storage uri rrrr Nurlr presign_ttl<) redirect_torzno-store, max-age= Cache-Controlbaggagez sentry-trace) resolve_storage_urir)r*r0r,rrr1getr HTTP_303_SEE_OTHERheaderspop) r5rrr6resolvedr7r=max_ageresponses r:r3z1ResolveStorageUriAPIMixin.redirect_to_presign_urlBs >33G LL9'% |STU]U`U`Taacdgchi j6#<#<= = >sC D+AD& D+&D+c #Ktj}tj}tj}||z}d}d} |j |D]S}tj} | |k\r!t jd|d|d|dn|dz }|t|z }|Utj|z } |jt jd | dd|d|dy #t$r$} t jd| d Yd } ~ qd } ~ wwxYw#t$r"} t jd | Yd } ~ wd } ~ wwxYw#tj|z } |jn.#t$r"} t jd | Yd } ~ nd } ~ wwxYwt jd | dd|d|dwxYww)zM Generator that stops yielding chunks after timeout seconds. r) chunk_sizez Time limit (zs) reached after yielding z chunks (z bytes)z%Error during time-limited streaming: Texc_infoNzCouldn't close stream: z!Stream processing finished after z.2fz s, yielded ) rRESOLVER_PROXY_BUFFER_SIZERESOLVER_PROXY_TIMEOUTtime monotonic iter_chunksr*warninglenr)r0closer+) r5 stream_bodyrLtimeout start_timedeadlinechunks_yielded total_byteschunk current_timeeelapseds r:time_limited_chunkerz.ResolveStorageUriAPIMixin.time_limited_chunkerZs88 11^^% '  $00J0G #~~/ 8+NN&wi/I.IYYbcnboovw!#s5z)   "nn&3G <!!# LL3GC= NK[[depdqqxy  U LL@DtL T T U  < 6qc:;; <nn&3G <!!# < 6qc:;; < LL3GC= NK[[depdqqxy s>G A(C2)G D"!G 2 D;DEDE" E +EG E  G G )E:9G : F%F G  F%%$G  G ctj}d}|jjdx}r"t |\}}|d}t j d|d| |dk(r |dk(s|dk(rnp|dkDr|dk(s|dk(r||z}n[|dk\r|dkDr|||zk\r||zn|}nA|dkr t jd|d}||z}nt jd|d}d}d |d |}t j d t|d z d z d t j dt|xsdd z d z d t j d||S)a Process and override Range header to limit stream size. This function does a trick: limit stream chunk sizes to MAX_RANGE, this way we free sync LSE workers for hanging too long, because the connection will be closed after the MAX_RANGE chunk is over. This function handles several range request formats: 1 'bytes=0-' and 'bytes=0-0': Passes through unchanged (header probes) 2 'bytes=123456-' and 'bytes=123456-0': Limits to MAX_RANGE bytes 3 'bytes=123456-789012': Limits the range if it exceeds MAX_RANGE 4 'bytes=-1024': Handles negative start (not supported) Returns: str: Modified range header in format "bytes=start-end" or None if no range header NRangez#>> Range read from request: start: z, end: rz+Start range is negative and not supported: zRange is not covered by logic: zbytes=-z>> stream > start: iz MBz>> stream > end: z>> stream > range_header: ) rRESOLVER_PROXY_MAX_RANGE_SIZErFrDrr*r+rUint)r5rmax_range_size range_headerrngstartends r:override_range_headerz/ResolveStorageUriAPIMixin.override_range_headers "?? //%%g. .3 .$S)JE3{ LL>ugWSER S zsbyC1Hr SAXn,!a03u~7M0Men,SV!LSERSn,!@FG#E7!C51L LL.s5z$t/C.DCH I LL,S]4-?-D,ESI J LL5l^D Ec|jdrt|d|jd<|jdr|d|jd<|jdrH|d}t|dr|j d|jd<nt||jd<d |jd <t j }d |d |jd <|j}t|j|}|j|} | |jd<|jdr)|jdxx|djdz cc<d|jdd|jd<|S)z2Prepare and set headers for the streaming response ContentLengthzContent-Length ContentRangez Content-Range LastModifiedstrftimez%a, %d %b %Y %H:%M:%S GMTz Last-Modifiedbytesz Accept-Rangeszprivate, max-age=z, must-revalidaterAETag") rDstrrFr2rtrRESOLVER_PROXY_CACHE_TIMEOUTr#rhr"r,strip) r5rJmetadatarr.last_modrIr# has_accessuser_status_tags r:prepare_headersz)ResolveStorageUriAPIMixin.prepare_headerssv << (14Xo5N1OH  - . << '080HH  _ - << '/Hx,4<4E4EFa4b  147M  1-4)77.?yHY,Z) ||//56 !WWIj\2&5%6 <<    V $(8(>(>s(C C $%&x'7'7'?&@#B roc |j|}|j||\}}}|6tjd|t dddt j S|rd|vrtj|\} } | r| }|j|} |d } t| |xsd | } |j| |||} tjr_d |jvrQ|jjd | jjdk(rt!t j"S| S#t$$rK}tjd|dt dt'|dt j cYd}~Sd}~wwxYw)a: Proxy the data using iter_chunks directly from storage streaming object. This implementation forwards Range headers to cloud storages and streams the response directly using StreamingHttpResponse. It avoids any intermediate buffering but doesn't support backward seeking. )rjNz)Failed to get direct stream from storage z)Storage stream failed while proxying datazStream is None)r0detailr z octet-stream StatusCodezapplication/octet-stream) content_typerrdz If-None-Matchrvz$Error in direct proxy from storage: TrN)rnget_bytes_streamr*r0rrHTTP_424_FAILED_DEPENDENCY mimetypes guess_typerbr rr RESOLVER_PROXY_ENABLE_ETAG_CACHErFrDrHTTP_304_NOT_MODIFIEDr)r)r5rurir.r9rjstreamrr{ guessed_type_time_limited_stream status_coderJr`s r:r4z1ResolveStorageUriAPIMixin.proxy_data_from_storages- 55g>L.5-E-EcXd-E-e *FL(~ H RSIUef!<< >\#A"+"6"6s"; a#/L#'";";F"C #<0K,#,2\B\epH ++HhQH88WGOO=[??&&78;K;K;O;OPV;WW'v/K/KLLO  LL?sCdL SEQ`abQcd88  s,AD1"C D1/D11 F:AF:FFN)r! __module__ __qualname__rrxrrr rr;r3rbrnrr4ror:rrsw&T{&TS&TE$PW-DX&T]e&TPseD'M>R`chp0$L=~#J5rorT)excludec"eZdZdZdgZefZdZy)TaskResolveStorageUrizA file proxy to presign storage urls at the task level. If the storage has presign=False, it will proxy the data through Label Studio instead of redirecting to presigned URLs. rDc||j}|jd}|jjd}||ttj S t jj|}|j|||S#t j$rttjcYSwxYw)LGet the presigned url for a given fileuri or proxy data through Label Studiotask_idrr pk) rrDGETrrHTTP_400_BAD_REQUESTrobjects DoesNotExistr1r;)r5rargskwargsrrtasks r:rDzTaskResolveStorageUri.get's,,**Y'++//), ?go6#>#>? ? ><<##w#/D||GWd33   >6#<#<= = > B -B;:B;Nr!rr__doc__http_method_namesrpermission_classesrDrror:rrs )+4rorc"eZdZdZdgZefZdZy)ProjectResolveStorageUrizA file proxy to presign storage urls at the project level. If the storage has presign=False, it will proxy the data through Label Studio instead of redirecting to presigned URLs. rDc||j}|jd}|jjd}||ttj S t jj|}|j|||S#t j$rttjcYSwxYw)r project_idrr r) rrDrrrrr rrr1r;)r5rrrrrr.s r:rDzProjectResolveStorageUri.getCs,,ZZ - ++//), ?j06#>#>? ? >oo))Z)8G||GWg66## >6#<#<= = >rNrrror:rr8s )+7ror))r%loggingrrRtypingr urllib.parsercore.utils.exceptionsr django.confr django.httprrr r drf_spectacular.utilsr projects.modelsr rest_frameworkrrest_framework.permissionsrrest_framework.responserrest_framework.viewsr tasks.modelsr"label_studio.io_storages.functionsrlabel_studio.io_storages.utilsr getLoggerr!r*rrrrror:rs   1 ^^/#!6,(A6   8 $@@Ft45w446t78'77ro