Project

General

Profile

Actions

Bug #6391

closed

osmo-epdg: crashs the UE fsm when receiving a GSUP Auth Request at the wrong time in state authenticating

Added by lynxis about 2 months ago. Updated about 2 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Target version:
-
Start date:
03/05/2024
Due date:
% Done:

100%


Description

16:13:51.838 [info] ue_fsm state_wait_swx_maa event=rx_swx_maa, {ue_fsm_data,"901702300008931",undefined,false,false,undefined}
16:13:51.838 [info] ue_fsm received_swm_auth_response {ok,{'MAA',"epdg.localdomain;1771156376;4;nonode@nohost",1,"hss.mnc070.mcc901.3gppnetwork.org","mnc070.mcc901.3gppnetwork.org",{'Vendor-Specific-Application-Id',10415,[16777265],[]},"901702300008931",[],[],[{'Experimental-Result',10415,5001}
],[],[],[],[],[],[],[],[]}}
16:13:51.838 [info] ue_fsm state_wait_auth_resp event=received_swm_auth_response, {ue_fsm_data,<<"901702300008931">>,"internet",undefined,false,0}
16:13:51.838 [info] auth_response(<<"901702300008931">>): {ok,{'MAA',"epdg.localdomain;1771156376;4;nonode@nohost",1,"hss.mnc070.mcc901.3gppnetwork.org","mnc070.mcc901.3gppnetwork.org",{'Vendor-Specific-Application-Id',10415,[16777265],[]},"901702300008931",[],[],[{'Experimental-Result',10415,5
001}],[],[],[],[],[],[],[],[]}}
16:13:51.838 [info] auth_response for <<"901702300008931">>: {ok,{'MAA',"epdg.localdomain;1771156376;4;nonode@nohost",1,"hss.mnc070.mcc901.3gppnetwork.org","mnc070.mcc901.3gppnetwork.org",{'Vendor-Specific-Application-Id',10415,[16777265],[]},"901702300008931",[],[],[{'Experimental-Result',1041
5,5001}],[],[],[],[],[],[],[],[]}}
16:13:51.838 [info] GSUP: Tx #{auth_tuples => [],imsi => <<"901702300008931">>,message_class => 5,message_type => send_auth_info_res}
Stream 238, 24 bytes
*DBG* gsup_server got {ipa,#Port<0.44>,
                           {osmo,5},
                           #{cn_domain => 1,
                             current_rat_type => rat_eutran_sgs,
                             imsi => <<"901702300008931">>,message_class => 5,
                             message_type => send_auth_info_req,
                             num_vectors_req => 1}}
*DBG* epdg_ue_fsm_901702300008931 receive call {auth_request,33,"*"} from <0.520.0> in state state_authenticating
16:13:57.306 [info] GSUP: Rx #{cn_domain => 1,current_rat_type => rat_eutran_sgs,imsi => <<"901702300008931">>,message_class => 5,message_type => send_auth_info_req,num_vectors_req => 1}
16:13:57.306 [info] ue_fsm auth_request
16:13:57.306 [info] terminating epdg_ue_fsm with reason function_clause state=state_authenticating, {ue_fsm_data,<<"901702300008931">>,"internet",undefined,false,0}
accept returned {error,closed} - goodbye!
16:13:57.307 [error] gen_statem epdg_ue_fsm_901702300008931 in state state_authenticating terminated with reason: no function clause matching state_functions
16:13:57.308 [error] CRASH REPORT Process epdg_ue_fsm_901702300008931 with 2 neighbours crashed with reason: no function clause matching epdg_ue_fsm:state_authenticating({call,{<0.520.0>,#Ref<0.716642772.2414084098.220027>}}, {auth_request,33,"*"}, {ue_fsm_data,<<"901702300008931">>,"internet",
undefined,false,0}) line 222
16:13:57.309 [error] Supervisor osmo_epdg_sup had child gsup_server started with gsup_server:start_link("127.0.0.1", 4222, []) at <0.520.0> exit with reason no function clause matching epdg_ue_fsm:state_authenticating({call,{<0.520.0>,#Ref<0.716642772.2414084098.220027>}}, {auth_request,33,"*"}
, {ue_fsm_data,<<"901702300008931">>,"internet",undefined,false,0}) line 222 in context child_terminated
16:13:57.310 [error] Supervisor osmo_epdg_sup had child aaa_diameter_swx started with aaa_diameter_swx:start_link() at undefined exit with reason no match of right hand value {error,already_started} in aaa_diameter_swx:init/1 line 134 in context start_error
16:13:57.310 [error] CRASH REPORT Process <0.714.0> with 0 neighbours crashed with reason: no match of right hand value {error,already_started} in aaa_diameter_swx:init/1 line 134
16:13:57.310 [error] Supervisor osmo_epdg_sup had child aaa_diameter_swx started with aaa_diameter_swx:start_link() at undefined exit with reason no match of right hand value {error,already_started} in aaa_diameter_swx:init/1 line 134 in context start_error
16:13:57.310 [error] CRASH REPORT Process <0.716.0> with 0 neighbours crashed with reason: no match of right hand value {error,already_started} in aaa_diameter_swx:init/1 line 134
16:13:57.311 [error] Supervisor osmo_epdg_sup had child aaa_diameter_swx started with aaa_diameter_swx:start_link() at undefined exit with reason no match of right hand value {error,already_started} in aaa_diameter_swx:init/1 line 134 in context start_error
16:13:57.312 [error] CRASH REPORT Process <0.718.0> with 0 neighbours crashed with reason: no match of right hand value {error,already_started} in aaa_diameter_swx:init/1 line 134
16:13:57.312 [error] Supervisor osmo_epdg_sup had child aaa_diameter_swx started with aaa_diameter_swx:start_link() at undefined exit with reason no match of right hand value {error,already_started} in aaa_diameter_swx:init/1 line 134 in context start_error
16:13:57.313 [error] CRASH REPORT Process <0.720.0> with 0 neighbours crashed with reason: no match of right hand value {error,already_started} in aaa_diameter_swx:init/1 line 134
16:13:57.313 [error] Supervisor osmo_epdg_sup had child aaa_diameter_swx started with aaa_diameter_swx:start_link() at undefined exit with reason no match of right hand value {error,already_started} in aaa_diameter_swx:init/1 line 134 in context start_error
16:13:57.313 [error] CRASH REPORT Process <0.722.0> with 0 neighbours crashed with reason: no match of right hand value {error,already_started} in aaa_diameter_swx:init/1 line 134
16:13:57.313 [error] Supervisor osmo_epdg_sup had child aaa_diameter_swx started with aaa_diameter_swx:start_link() at {restarting,undefined} exit with reason reached_max_restart_intensity in context shutdown

Files

osmo-epdg-crash.pcapng osmo-epdg-crash.pcapng 11.2 KB lynxis, 03/05/2024 03:39 PM
Actions #1

Updated by lynxis about 2 months ago

I think there are two issues here:
- the MAR is responding with exp Result 5001 (User Unknown)
- The second GSUP Request arrives in state authenticating, which can't handle gsup auth requests.

EDIT: In the pcap the last packet is crashing the osmo-epdg.

Actions #2

Updated by pespin about 2 months ago

Result IMSI_UNKNWON reproduced here:
https://gerrit.osmocom.org/c/osmo-ttcn3-hacks/+/36172 epdg: Introduce test TC_authinfo_MAA_unknown_user

I'll be working on this tomorrow, the MAR+MAA procedures are not as robust as others in current code.

Actions #3

Updated by pespin about 2 months ago

lynxis fyi running a successful test after the failing one still works:

EPDG_Tests.TC_authinfo_MAA_unknown_user
EPDG_Tests.TC_authinfo_normal

Which means in general osmo-epdg keeps working after the failure. It may be though that the specific subscriber FSM ends up in some weird state so the service won't work for that specific one.
I'll be fixing all this tomorrow.

Actions #4

Updated by pespin about 2 months ago

  • Status changed from New to In Progress
Actions #5

Updated by pespin about 2 months ago

  • % Done changed from 0 to 50

I also tested the problem about re-sending GSUP AuthInfoReq here:
https://gerrit.osmocom.org/c/osmo-ttcn3-hacks/+/36178 epdg: Introduce test TC_authinfo_twice

Fixed in osmo-epdg here:
remote: https://gerrit.osmocom.org/c/erlang/osmo-epdg/+/36176 gsups_server: Monitor epdg_ue_fsm instead of linking it [NEW]
remote: https://gerrit.osmocom.org/c/erlang/osmo-epdg/+/36177 epdg_ue_fsm: Allow UE restarting authentication

TODO: I now need to fix osmo-epdg to pass TC_authinfo_MAA_unknown_user (it's not propagating the error from SWx MAA to GSUP).

Actions #6

Updated by pespin about 2 months ago

  • Status changed from In Progress to Feedback
  • % Done changed from 50 to 90

Forwarding of error fixed here:
https://gerrit.osmocom.org/c/erlang/osmo-epdg/+/36180 Forward failure SWx MAA -> GSUP AuthInfo Error

Actions #7

Updated by pespin about 2 months ago

  • Status changed from Feedback to Resolved
  • % Done changed from 90 to 100

Patches merged, closing ticket.

Actions

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 48.8 MB)