[MAC-RRAS(VPN)] - "Negotiation Timed Out" for Always-On VPN (IKEv2)

Hello Sujith,

I think that you need to persevere with racoon logging first. Other layers possible report failures by lower layers in a misleading way.

Look at section 2.16 of RFC 5996:

   Initiator                         Responder
   -------------------------------------------------------------------
   HDR, SAi1, KEi, Ni  -->
                                <--  HDR, SAr1, KEr, Nr, [CERTREQ]
   HDR, SK {IDi, [CERTREQ,]
       [IDr,] SAi2,
       TSi, TSr}  -->
                                <--  HDR, SK {IDr, [CERT,] AUTH,
                                         EAP }
   HDR, SK {EAP}  -->
                                <--  HDR, SK {EAP (success)}
   HDR, SK {AUTH}  -->
                                <--  HDR, SK {AUTH, SAr2, TSi, TSr }

I am fairly sure that the last packet (which was sent by the server) was HDR, SK {IDr, [CERT,] AUTH, EAP }; the EAP attribute is just an EAP "start" message. The attribute most likely to have been found unacceptable by the macOS client is the server CERT.

Gary

could you assist me setting up racoon logging further? please share a action plan so that I can perform on my mac client.

FYI, I even think the same, server is requesting for certificate from the client and the client does not OR unable to fulfill the requested action, yes, I should dig deeper to see why mac not responding to IKE_AUTH OR what is missing

Please assist me here with your action plan.

Sujith

So you say server CERT is the problem, which one you are referring to SCEP User CERT OR Root CERT or VPN Cert?

FYI, we have our PKI and SCEP user certs are distributed via NDES & Trusted Root CERT are distributed using Intune. The same procedure is followed for windows devices and they are getting connected without a problem.

Hello Sujith,

As mentioned, I don't currently have access to a Mac; it is difficult to describe steps that I can't test and have not performed for a long time.

The certificate in the last message is the certificate for the VPN server; the checks performed by macOS will require the whole chain to the root certificate and the root certificate must be trusted by macOS.

Gary

Hi Gary,

Thanks for your replies so far.

All the relevant certificates are set to "Always Trust' for EAP & IPSEC therefore I don't think there will be a problem with the trust factor but your statement makes me think about VPN server certificate, the VPN connection sting name is same as VPN server and the CERT was issued by the internal PKI but not signed by external CA. do you think this could also be problem for the mac to trust such certs?

the same is used on windows devices and there is no problem with it.

Sujith

Like mentioned in the following link http://glozer.net/soekris/macosx-ipsec.html , there is a script that creates an IPSec tunnel which prevents a potential sniffer from analyzing network traffic based on the final destination of the packet, do you think running this script might give us more details what's happening in the backend while initiating connection. moreover I have also attached the racoon.cong "https://1drv.ms/u/s!AkVuXnM-kirQkPpbWJ2GcB_zmTmulg?e=7KORj4" from /etc/racoon/racoon.conf file fyi.

please suggest here if we can take this direction to check more insight logs from mac.

There is also one another thing that runs in my mind, The IPsec policy must match on both the server and the client for an IKEv2 VPN connection to be successful, on windows we have PowerShell commands to set the custom IPSEC policies but not sure if it is required to be set on macOS as well.

my VPN server's IPSEC policy is as follows:

• AuthenticationTransformConstants SHA256128
• CipherTransformConstants AES128
• DHGroup Group14
• EncryptionMethod AES128
• IntegrityCheckMethod SHA256 -
• PFSgroup PFS2048
• SALifeTimeSeconds 28800
• MMSALifeTimeSeconds 86400
• SADataSizeForRenegotiationKilobytes 1024000

Is it really needed to set the above ipsec policy on mac client as well? if so where this should be done, is it in the raccon.conf or it should be the part .mobileconfig file.

Regards
Sujith

Hello Sujith,

As I have already, mentioned, setting a higher logging level on racoon might help.

The .conf file that you posted includes the following lines:

# "log" specifies logging level. It is followed by either "notify", "debug"
# or "debug2".
#log debug;

The "#" symbol means comment. If you were to add "log debug2" to the .conf file, then there should be more useful information in the logs; since there is currently no active "log" directive, we are probably just seeing the default (low) level of logging.

Gary

Hi Gary

As you mentioned, I added the log level to "log debug2" and tried the below command along with --last switch to get the latest logs but unfortunately it returns result only when I specify --last 6h (which is 6 hours)

log show --debug --predicate 'process == "racoon"' --last 6h    // gives results but older than 6 hours log

tried below

 log show --debug --predicate 'process == "racoon"' --last 5m // no results returned when specified 5 minutes 

Alongside in search for syntax, I came across this link https://www.daemon-systems.org/man/racoon.conf.5.html where they talk about how to configure the racoon config file. Is it really mandatory to modify authentication_algorithm, encryption_algorithm, remotehostname, radius, PH Group etc? I guess, during the initial packet exchange by default mac adapts to payloads exist in VPN server so there is no need to match between client and server?

correct me if I am wrong with my assumption mentioned above.

Regards
Sujith

Hello Sujith,

It might be necessary to send a signal to racoon (or to restart racoon) in order for changes in racoon.conf to take effect.

Once that has been done, you could try log show --debug without the 'process == "racoon"' predicate but selected by time (e.g. --last or --start or whatever) and make that available.

Mutually acceptable algorithms and parameters have to be negotiated for quick mode and main mode. The traces show that quick mode negotiation was successful (the encrypted packets in the trace are using the negotiated algorithms/parameters). Main mode negotiation has not yet completed; until the Mac is able/willing to send the next packet, we won't know for sure whether any configuration adjustments in this regard are necessary.

Gary

HI Gary

Thanks for the clarity thrown to the question about initial handshake I asked in my previous question.

Yesterday itself, I tried restarting racoon using the below command still it does not show up the latest log entries.

sudo launchctl stop com.apple.racoon

sudo launchctl start com.apple.racoon

still finding a way to get the latest logs.

Regards
Sujith

After a long hunt on getting detailed information about IKE logs, I identified that my mac does not trust the certificate issued by an Internal CA and the reason is unknown, due to which mac failed to use the certificate for authentication. please see the screenshot below for your reference.

Now, i need to figure out why these certificates are not trusted even after setting it to TRUST ALWAYS for ALL USERS from the keychain app.

iRegards
Sujith

Hello Sujith,

I think that one should focus on establishing the root certificate authority as a trusted root authority. I don't think that any special steps are needed for the specific certificate of the VPN server,

Gary

Hello Sujith,

A potential problem might be that the VPN server certificate contains a CRL Distribution Points extension that only lists "unreachable" (for the Mac) distribution points (e.g. that use the ldap:/// scheme).

Gary