Advanced Authentication Flows

How to configure your application log-in and HTTP Basic Authentication (including advanced authentication flows such as Parameter based Authentication, OAuth2, or SAML)

What is Application Authentication?

Many web applications rely on some login mechanism to authenticate users. A full security scan of the application is in most cases only useful if the whole application (including all internal areas) is scanned. This article explains the different authentication methods to enable a scan of the whole web application:

  • Form-based authentication
    • HTTP Basic Authentication
    • Form-Based Login 
  • Advanced Authentication Methods
    • Parameter Authentication 
    • Scripting Authentication Flows / Automating authenticated scans with Crashtest Security. 
    • OAuth2 
    • SAML 
  • Testing for privilege escalation  

If you would like to get more guidance on setting up an API for scans, please check out this wiki article.

Simple Authentication Methods

 

HTTP Basic Authentication

HTTP Basic authentication (also known as .htaccess protection) is an authentication method where an authorization header with a base64 encoded username and password is sent to the server. 

If an HTTP Basic authentication protects your web application, please use the global authentication setting in your scan target to configure the authentication by adding a username and Password valid for your .htaccess form.

HTTP Basic Authentication

Form-based authentication / Standard Login Form 

If your application uses a standard HTML-based login form, you can use the application authentication table to create a working log-in for your application. To configure this correctly, you have to follow three simple steps: 

  1. Enter the Log-In URL in the Application Login URL field. The URL should ideally only contain the log-in form and should be a separate URL; if this is not the case,   please have a look at the edge cases at the end of this chapter  
  2. Click the Button Create Profile and create a new authentication profile 
  3. Click on the newly created profile on the left (you might have to do a refresh) 
  4. Enter User Name and Password valid for the application 

Application-Authentication

Preferences-Authentication-Profile

We understand that no authentication set is the same; hence we have a collection of edge cases we have encountered in the past to make it easier for you to configure the scans. Please have a read through them; in case you cannot find a solution for your set-up, please connect with our security experts (https://meetings.hubspot.com/felix28/exchange-regarding-crashtest-security-suite): 

  • Modal / Pop-up Log-in
    • Ideally, a specific subpage (e.g., www.example.com/login) contains the log-in form, or you can make one available, as then our crawler will log in more easily. 
    • If this is not the case, you can run a test run, selecting the URL to click the button to open the log-in. In some cases, the crawler identifies the log-in correctly, and an authenticated scan can be conducted. 
    • In Case this is not possible, please refer to parameter authentication 
  • Log-In Form is present on the main site 
    • Please configure the main page as a log-in URL and start the scan.
    • If the log-in occurs in a modal, please refer to the steps mentioned here 
    • In Case this is not possible, please refer to parameter authentication
  • Cookie Banners 
    • If you use a Consent Management Tool like (Cookiebot, ..) on the application to be tested, please either: 
    • Disable the Consent Management Tool for the time the scan is running 
    • Or configure a token in the Global parameter authentication settings that disable the CMT to pop-up (e.g., consent_cookie=1) 
  • Additional input fields that are required for a log-in 
    • Our crawler can detect additional input fields on the log-in form as follows: 
      • Checkboxes (e.g., Agreement to T&’C’s) will be checked if they are configured as checkboxes in the code 
      • Drop-Down Menus: The Crawler will always select the first choice in a drop-down menu; please make sure that the Credentials provided match the selection 
      • Free-text fields: Please make sure that these are prefilled with correct values when visiting the site. If they are not, the crawler will not fill in any value, and the log-in might not be successful. In this case, please refer to parameter authentication
  • 2-Factor Authentication 
    • If your web application uses some 2-factor authentication (e.g., SMS, authenticator app, etc.), we will currently not perform a log-in with a second factor. To still use the scanner successfully, you can use the following options: 
      • Create a profile with users that do not have 2FA enabled and configure the log-in accordingly 
      • Disable the 2FA on the test/staging environment or a dedicated pentesting system 
      • For some cases (e.g., token-based authentication), there is potentially a way to bypass the 2FA. Please consult the support for help. 
  • Dynamically changing URLs 
    • In case your web application creates a dynamic log-in URL every time a user attempts a log-in, there are the following options for you to configure the login: 
      • Create a page that automatically directs the user to the newly created log-in site; configure this in the log-in URL. Our scanner will follow the redirect and log in with the provided credentials on 
      • Create a static log-in page 
      • Refer to parameter authentication 
  • SSO Log-In 
    • See below 

Parameter authentication

This setting allows you to configure a more advanced authentication and should be used if the form-based authentication does not work. There are three main types of configuration:

  • HTTP Header: If you configure an HTTP header, the header will be sent with each request used for scanning the web application
  • GET parameter: If you configure a GET parameter, the parameter will be added to each request as GET parameter when scanning the web application
  • (Session) Cookies: If you configure a cookie, the cookie will be sent with each request used for scanning the web application

Important points for the parameter authentication: 

  • Please make sure that the lifetime for tokes/cookies is long enough for the scan to run through completely (we recommend at least 24hrs) 
  • For Javascript-based applications: Please make sure you configure all additional information your application uses to verify authenticated users. This can include
    • Session storage
    • Local storage
    • Custom Cookies / Tokens 
  • Depending on your setup, you can configure parameter authentication on a global or per-user profile level. However, please do not configure it in the global settings and the user settings, leading to conflicting results. 
  • Currently, we do not support cookies/tokens/headers to be renewed during the scan. However, they can be generated automatically before every scan (please check the next chapter) 
  • Currently, we are working on an option to verify the authentication before starting a scan 

How to configure parameter authentication: 

  1. You choose the parameter type (for an explanation, see above) 
  2. You can give the parameter a Title/identifier, so other users in your team understand its meaning. 
  3. Parameter Key: please configure the exact key/phrase used by your application for the authentication. For example, this could be, e.g., “Authorization,” “Authentication,” or your own customer identifier. 
  4. Parameter Value: Please configure the exact value used by your application for the authentication. 
    1. An authorization header should usually contain “Bearer (unique Identifier).”
    2. A cookie should usually contain the attributed session ID

Parameter Authentication

Scripting Authentication Flows

It is possible to send a payload via Webhook or API, which contains authentication data. This data will be used to configure your project before starting the scan. Please note that previous user credentials will be overwritten, and the credentials will be stored for the next scan. To send a webhook request with payload data, you may use the following request:

curl --data '{
"system_authentication": {
"basic_auth": {
"username": "username",
"password": "password"
}
},
"application_authentication": [
{
"username": "username",
"password": "password",
"url": "example.com/login"
}
],
"parameter_authentication": [
{
"type": "HEADER",
"key": "Authentication",
"value": "Bearer 12345678"
}
]
}' -X POST https://api.crashtest.cloud/webhook/SECRET_HASH

Manual Login Flow

Let's assume that you have a REST API at https://example.com/api/v1, which uses JWT Tokens for authentication. The token is generated by a POST request to /login with the username 'username' and the password 'password.'

$ curl --data '{"username": "username", "password": "password"}' -X POST https://example.com/api/v1/login

{
"token_type": "Bearer",
"expires_in": 86400,
"access_token": "ABCDEF",
"refresh_token": "ABCDEF",
"concurrent_sessions": 1,
"max_current_sessions": 40,
}

You may use this to generate a script that logs you in before starting the security scan. The token generated by the login request is then used for the security scan.

ACCESS_TOKEN=`curl --data '{"username": "username", "password": "password"}' -X POST https://example.com/api/v1/login | jq -r '.access_token`
curl --data "{
\"parameter_authentication\": [
{
\"type\": \"HEADER\",
\"key\": \"Authentication\",
\"value\": \"Bearer $ACCESS_TOKEN\"
}
]
}" -X POST https://api.crashtest.cloud/webhook/SECRET_HASH

Responses

The following JSON responses inform you about the result of the webhook call:

{"message": "webhook_scan_started", "data": {"scanId": SCAN_ID}}    # Success Case
{"message": "Scan is already running"} # Failure Case

Advanced Authentication Methods

You need to implement the login using a custom script to retrieve the authentication information for the advanced authentication methods, such as a session ID stored in a cookie or a JWT token.

SAML

SAML (Security Assertion Markup Language) is an XML Framework to exchange authentication and authorization information. When using a SAML workflow, you need a script (CLIENT) to log in using your identity provider (IDP) and generate a session with the application that shall be scanned, the service provider (SP), and the handover of the credentials to the Crashtest Security Suite.

When writing your login script, you need to handle the following steps:

  1. First, CLIENT visits the IDP login page for the SP.
  2. Next, CLIENT provides credentials to the IDP.
  3. IDP generates the SAML response and sends it to the CLIENT
  4. CLIENT posts the SAML response message to the SP
  5. SP authenticates the user based on the assertion and provides session information (e.g., session cookie)
  6. CLIENT sends a request to the Crashtest Security Suite to start a security scan, including the session information 

To check out how to send the information to the Crashtest Security Suite (step 6), look at the Using Webhooks article. You can also set the session manually for a project:

Screenshot 2020-04-02 at 18.38.05

OAuth 2

OAuth 2 (Open Authorisation) is a protocol that offers a secure and standardized API authentication. With an OAuth 2.0 workflow, a client gets an access token from an authorization server (AS) used to authenticate with the software that shall be scanned. For OAuth 2, there exist several different authentication flows that can be used. A simple one is the Resource Owner Password Credentials Grant flow. Create a login script (CLIENT) that does the following:

  1. CLIENT sends a request to the AS to generate an access token
  2. AS sends an access token to the CLIENT
  3. CLIENT sends a request to the Crashtest Security Suite to start a security scan, including the access token

To check out how to send the information to the Crashtest Security Suite (step 6), look at the Using Webhooks article. You can also set the session manually for a project.

If you have any questions, for example, on how to handle Single-Sign-On (SSO) authentication, please don't hesitate to contact us.