]jaL v d dl Z d dlmZ d dlmZ d dlmZ d dlmZ d dl m
Z
d dl m Z e Z
G d d Zy)
N)get_user_model) LabelStudio)OrganizationFactory)ProjectFactory) TaskFactory) UserFactoryc e Zd ZdZ ej
d d Zd Zd Zd Z d Z
d Z d
Z d Z
d Zd
Zd Zd Zd Zd Zd Zd Zd Zd Zd Zd Zd Zy)TestSDKPredictionValidationzDComprehensive tests for prediction validation using Label Studio SDKT)autousec |j 5 t | _ t | j | _ | j | j _ | j j
t dd| j | j | _ t | j ddi | _
ddd y# 1 sw Y yxY w) zRSet up test environment with user, organization, project, and task using factories)
created_byz Test Projecta
)title label_configr
organizationtextz)John Smith works at Microsoft in Seattle.)projectdataN) unblockr userr r active_organizationsaver r r task)selfdjango_db_setupdjango_db_blockers [/root/env/lib/python3.12/site-packages/label_studio/tests/sdk/test_prediction_validation.pysetupz!TestSDKPredictionValidation.setup s
&
&
( % v#
DI 3tyy ID ,0,=,=DII ) IINN *$ . 99 ! . .5 DL< $DLLHs?tuDIK% v % v % vs B&C C c @ t ||j }| j j dddddgidgddd } |j j
d i |}|j J |j | j j k( sJ |j |d k( sJ y
)
z7Test creating a valid prediction with choices using SDKbase_urlapi_key sentimentr choicespositive from_nameto_nametypevalueffffff?v1.0r resultscore
model_versionNr- r r! r id predictionscreater- r django_live_urlbusiness_clientlsprediction_data
predictions r test_valid_prediction_choicesz9TestSDKPredictionValidation.test_valid_prediction_choices9 s
/?;R;R
S IILL )fi[dgqfrZst #
+R^^**=_=
}}((($)),,... OH$==== c F t ||j }| j j ddddgdddd d
gd d d
} |j j
di |}|j J |j | j j k( sJ |j |d k( sJ y)z6Test creating a valid prediction with labels using SDKr entitiesr labelspersonr
John Smithr? startendr r% 333333?r+ r, Nr- r0 r1 r5 s r test_valid_prediction_labelsz8TestSDKPredictionValidation.test_valid_prediction_labelsN s
/?;R;R
S IILL ", % $)1
QrS_ ` #
+R^^**=_=
}}((($)),,... OH$====r< c > t ||j }| j j dddddidgddd } |j j
d i |}|j J |j | j j k( sJ |j |d k( sJ y
)
z6Test creating a valid prediction with rating using SDKr qualityr rating r% ?r+ r, Nr- r0 r1 r5 s r test_valid_prediction_ratingz8TestSDKPredictionValidation.test_valid_prediction_ratingf s
/?;R;R
S IILL%.68_gij^klm #
+R^^**=_=
}}((($)),,... OH$====r< c @ t ||j }| j j dddddgidgddd } |j j
d i |}|j J |j | j j k( sJ |j |d k( sJ y
)
z8Test creating a valid prediction with textarea using SDKr summaryr textareaz This is a summary of the text.r% )\(?r+ r, Nr- r0 r1 r5 s r test_valid_prediction_textareaz:TestSDKPredictionValidation.test_valid_prediction_textareaw s
/?;R;R
S IILL "+ % & $'G&H I #
+R^^**=_=
}}((($)),,... OH$====r< c t ||j }| j j ddigddd}t j
t 5 |j j di | ddd y# 1 sw Y yxY w) z6Test prediction with missing required fields using SDKr r& r" r* r+ r, Nr0 r r! r r2 pytestraises Exceptionr3 r4 r r6 r7 r8 r9 s r test_missing_required_fieldsz8TestSDKPredictionValidation.test_missing_required_fields s{
/?;R;R
S IILL #
]]9
% 5 !BNN ! ! 4O 4 5 5 5s A55A>c t ||j }| j j dddddgidgddd }t j
t 5 |j j d i | d
d
d
y
# 1 sw Y y
xY w) z5Test prediction with non-existent from_name using SDKr nonexistent_tagr r# r$ r% r* r+ r, Nr0 rT rX s r test_invalid_from_namez2TestSDKPredictionValidation.test_invalid_from_name s
/?;R;R
S IILL "3 % % '* 6 #
]]9
% 5 !BNN ! ! 4O 4 5 5 5 A;;Bc t ||j }| j j dddddgidgddd }t j
t 5 |j j d i | d
d
d
y
# 1 sw Y y
xY w) z3Test prediction with non-existent to_name using SDKr r" nonexistent_targetr# r$ r% r* r+ r, Nr0 rT rX s r test_invalid_to_namez0TestSDKPredictionValidation.test_invalid_to_name s
/?;R;R
S IILL "- 3 % '* 6 #
]]9
% 5 !BNN ! ! 4O 4 5 5 5r] c t ||j }| j j dddddgidgdd d
}t j
t 5 |j j d i | d d d y # 1 sw Y y xY w)
z,Test prediction with type mismatch using SDKr r" r r? r# r$ r% r* r+ r, Nr0 rT rX s r test_type_mismatchz.TestSDKPredictionValidation.test_type_mismatch s
/?;R;R
S IILL "- % $ '* 6 #
]]9
% 5 !BNN ! ! 4O 4 5 5 5r] c t ||j }| j j dddddgidgddd }t j
t 5 |j j d i | d
d
d
y
# 1 sw Y y
xY w) z3Test prediction with invalid choice value using SDKr r" r r# invalid_choicer% r* r+ r, Nr0 rT rX s r test_invalid_choice_valuez5TestSDKPredictionValidation.test_invalid_choice_value s
/?;R;R
S IILL "- % % '*:); < #
]]9
% 5 !BNN ! ! 4O 4 5 5 5r] c t ||j }| j j dddddidgddd }t j
t 5 |j j d i | d
d
d
y
# 1 sw Y y
xY w) z3Test prediction with invalid rating value using SDKr rI r rJ r% rL r+ r, Nr0 rT rX s r test_invalid_rating_valuez5TestSDKPredictionValidation.test_invalid_rating_value s
/?;R;R
S IILL "+ % $ &] #
]]9
% 5 !BNN ! ! 4O 4 5 5 5 A::Bc t ||j }| j j ddddgdddd d
gd d d
}t j
t 5 |j j di | ddd y# 1 sw Y yxY w)z3Test prediction with invalid labels value using SDKr r> r r?
invalid_labelr rA rB rC r% rF r+ r, Nr0 rT rX s r test_invalid_labels_valuez5TestSDKPredictionValidation.test_invalid_labels_value s
/?;R;R
S IILL ", % $#2"3!" ! ,
#!
( ]]9
% 5 !BNN ! ! 4O 4 5 5 5 A>>Bc t ||j }| j j ddddgddd}t j
t 5 |j j d
i | d d d y # 1 sw Y y xY w) z2Test prediction with missing value field using SDKr r" r r# )r& r' r( r* r+ r, Nr0 rT rX s r test_missing_value_fieldz4TestSDKPredictionValidation.test_missing_value_field, s
/?;R;R
S IILL "- % % #
]]9
% 5 !BNN ! ! 4O 4 5 5 5s A77B c . t ||j }| j j g ddd} |j j
di |}|j J |j | j j k( sJ |j |d k( sJ y)z1Test prediction with empty result array using SDKr r* r+ r, Nr- r0 r1 r5 s r test_empty_result_arrayz3TestSDKPredictionValidation.test_empty_result_arrayB s
/?;R;R
S#'99<<2_ef*R^^**=_=
}}((($)),,... OH$====r< c
: t ||j }| j j dddddgiddddd ggd
d d d
ddddddgidgddd}t j
t 5 |j j di | ddd y# 1 sw Y yxY w)zYTest prediction with multiple regions where some are valid and some are invalid using SDKr r" r r# r$ r% r> r? r@ r rA rB rC r[ rF r+ r, Nr0 rT rX s r $test_multiple_regions_mixed_validityz@TestSDKPredictionValidation.test_multiple_regions_mixed_validityM s
/?;R;R
S IILL )fi[dgqfrZst!+ % $*2 qUa b "3 % % '* 6 #%
, ]]9
% 5 !BNN ! ! 4O 4 5 5 5s + BBc t ||j }| j j dddddgiddddd gd
d d d
dddddgdddd
ddddddiddddddgidgddd} |j j
d i |}|j J |j | j j k( sJ |j |d k( sJ y) z?Test a complex valid prediction with multiple regions using SDKr r" r r# r$ r% r> r? r@ r rA rB rC r MicrosoftrI rJ rK rO rP z"A person works at an organization.gq=
ףp?zv2.0r, Nr- r0 r1 r5 s r test_complex_valid_predictionz9TestSDKPredictionValidation.test_complex_valid_predictionj s
/?;R;R
S IILL )fi[dgqfrZst!+ % $)1
QrS_ ` ", % $)7(82bZe f (FHX`bcWde!* % & $'K&L M , #3
8 +R^^**=_=
}}((($)),,... OH$====r< c t ||j }| j j dddddidgddd }t j
t 5 |j j d i | d
d
d
y
# 1 sw Y y
xY w) z?Test prediction with invalid textarea value structure using SDKr rO r rP zThis should be a listr% rQ r+ r, Nr0 rT rX s r test_invalid_textarea_valuez7TestSDKPredictionValidation.test_invalid_textarea_value s
/?;R;R
S IILL "+ % & $&= > #
]]9
% 5 !BNN ! ! 4O 4 5 5 5ri c t ||j }| j j dddddddd d
gd d d
}t j
t 5 |j j di | ddd y# 1 sw Y yxY w)z7Test prediction with invalid labels structure using SDKr r> r r? r@ r rA rB rC r% rF r+ r, Nr0 rT rX s r test_invalid_labels_structurez9TestSDKPredictionValidation.test_invalid_labels_structure s
/?;R;R
S IILL ", % $"*!" ! ,
#!
( ]]9
% 5 !BNN ! ! 4O 4 5 5 5s A==Bc t ||j }| j j ddddggdddd gd
d d }t j
t 5 |j j di | d
d
d
y
# 1 sw Y y
xY w)z;Test prediction with missing text field in labels using SDKr r> r r? r@ r rA )r? rD rE r% rF r+ r, Nr0 rT rX s r test_missing_text_in_labelsz7TestSDKPredictionValidation.test_missing_text_in_labels s
/?;R;R
S IILL ", % $$,:,!" !
#!
( ]]9
% 5 !BNN ! ! 4O 4 5 5 5rm c t ||j }| j j ddddggdddd d
gd d d
}t j
t 5 |j j di | ddd y# 1 sw Y yxY w)z:Test prediction with invalid start/end positions using SDKr r> r r? r@ d n rB rC r% rF r+ r, Nr0 rT rX s r test_invalid_start_end_positionsz r r? r@ rA rB rC r% rF r+ r, Nr0 rT rX s r test_end_before_startz1TestSDKPredictionValidation.test_end_before_start s
/?;R;R
S IILL ", % $*2 r!Ua b #
]]9
% 5 !BNN ! ! 4O 4 5 5 5r N)__name__
__module__ __qualname____doc__rU fixturer r; rG rM rR rY r\ r` rb re rh rl ro rq rs rx rz r| r~ r r r0 r< r r
r
s NV^^D!'v "'vR>*>0>">05(5,5,5,5,5,565, >5:$>L5,5656565r< r
)rU django.contrib.authr label_studio_sdkr organizations.tests.factoriesr projects.tests.factoriesr tasks.tests.factoriesr users.tests.factoriesr Userr
r0 r< r r s.
. ( = 3 - -5 5r<