CodeWatch

Last week we finished up input validation, which represents one half of the solution towards preventing XSS. This week we will cover the other half; output encoding. Output encoding protects the application in the event that malicious script data somehow makes its way into the database or a form parameter. It is a fail safe if you will.

Let’s begin…

ASVS 6.1 Requirement:

Verify that all untrusted data that are output to HTML (including HTML elements, HTML attributes, javascript data values, CSS blocks, and URI attributes) are properly escaped for the applicable context.

ASVS 6.1 Solution:

Escaping or encoding data prior to output to the browser will help to ensure that malicious code is not executed, and is instead rendered as text. PHP provides two functions for encoding HTML data; htmlentities and htmlspecialchars. We will use htmlentities, as it encodes all characters for which there are HTML character entity equivalents (thorough). The htmlspecialchars function operates as its name suggests; only certain designated special/dangerous characters are encoded (&, ‘, “, <, >).

Example:

  $data = htmlentities($_POST['value'], ENT_QUOTES, "UTF-8", false);
  print("$data");

It is just as simple as that. Here we are assigning the POSTed form value of ‘value’ to the $data variable in an encoded form and later printing/outputting it to the browser. The first value passed to htmlentities is the data to be encoded. The following constant tells htmlentities what to do with quotes; ENT_QUOTES provides the most secure mechanism, which is to encode single AND double quote characters. The third variable passed to htmlentities sets the character set to be used. The final variable tells htmlentities whether or not to double encode (encode an already encoded value). Double encoding is unnecessary and could modify how the data is displayed.

This same function should be used to encode user supplied data that is being pulled from a database, xml file, etc. All user supplied data should be output encoded regardless of its origin.

ASVS 6.2 Requirement:

Verify that all output encoding/escaping controls are implemented on the server side.

ASVS 6.2 Solution:

This is an easy one, don’t escape/encode data on the client side as this can be bypassed. Use the htmlentities function within your server side PHP code to output encode user supplied data.

ASVS 6.3 Requirement:

Verify that output encoding /escaping controls encode all characters not known to be safe for the intended interpreter.

ASVS 6.3 Solution:

This was covered above in our solution for 6.1. Use htmlentities instead of htmlspecialchars, and you will be certain to encode everything with an HTML entity equivalent.

Note: We are skipping ASVS 6.4 through 6.8 as they cover escaping for SQL, XML, LDAP and other types of injection that will be reviewed in later blog posts.

ASVS 6.9 Requirement:

Verify that for each type of output encoding/escaping performed by the application, there is a single security control for that type of output for the intended destination.

ASVS 6.9 Solution:

Do not vary your security controls depending upon the destination (SQL database, LDAP, XML, etc). Perform the same type of output encoding for all user supplied data sources and regardless of the destination of the data. This will prevent malicious script code from later being displayed in some way, possibly by a different application.

ASVS 6.10 Requirement:

Verify that all code implementing or using output validation controls is not affected by any malicious code.

ASVS 6.10 Solution:

Scan your code for viruses, or any malicious backdoors. Perform manual and automated code reviews.

And that is it for output encoding. This is a very simple control to implement that will go a long way towards protecting your web applications from XSS attacks.

We have completed the sections in the OWASP ASVS covering controls to prevent XSS attacks. However, the next post will provide an input validation follow up that provides a couple additional security layers to help protect your applications. After that post, we will move on to a new OWASP 2010 Top 10 section.

As always, please provide any feedback you have, positive or negative.

Today we will finish up ASVS section 5. Next week we will begin ASVS 6.x (Output Encoding/Escaping Requirements). This should be a pretty short post.

ASVS 5.6 Requirement:

Verify that a single input validation control is used by the application for each type of data that is accepted.

ASVS 5.6 Solution:

Watch out for data validation code sprawl. Use the same data validation code for the same data type each time you perform the validation. Don’t rewrite or add the validation code to each page of PHP. Write it once as a function or class and then include that function/class in the pages requiring validation.

ASVS 5.7 Requirement:

Verify that all input validation failures are logged.

ASVS 5.7 Solution:

Record validation failures to a log file, database table, or both. This will help in performing forensics if the site is compromised as well as in responding to incidents/attacks as they occur. I will use the whitelist validation function from last week as an example, removing the comments from that demonstration:

  function dataValidator($type, $data) {
    $decodeData = html_entity_decode($data, ENT_QUOTES, 'UTF-8');

    $typeArray = array(
      "number" => '/^[0-9]+$/',
      "letter" => '/^[a-zA-Z]+$/',
      "alphan" => '/^[a-zA-Z0-9]+$/'
    );

    if (preg_match($typeArray[$type], $decodeData)) {
      return true;
    } else {
      // Pseudo code for calling a logging function.
      // Passing the date/time, type to be tested, 
      // as well as the data that was passed (encoded).
      writeLog(date('Y-m-d H:m:s') . 
        " - $type Validation failed with the following data: " . 
        htmlentities($decodeData, ENT_QUOTES, UTF-8, false));
      return false;
    }
  }

 
ASVS 5.8 Requirement:

Verify that all input data is canonicalized for all downstream decoders or interpreters prior to validation.

ASVS 5.8 Solution:

Data that is encoded in a special character set can bypass validation filters or be used in directory traversal attacks in many cases. Canonicalization enforces a specific character set and encodes the data (or modifies the file path) in a way that reduces or eliminates the risk of XSS or directory traversal attacks. This is accomplished through setting a default character set, encoding data, and canonicalizing file paths. Configure the default character set for PHP by adding/modifying this in php.ini:

  default_charset = "ISO-8859-1"

 
User supplied data which contains paths can be canonicalized with the realpath() PHP function:

  realpath($data);

 
User supplied data can be encoded into HTML characters using htmlentities or htmlspecialchars. The HTML entities function encodes everything for which there is an HTML character entity equivalent, whereas htmlspecialchars only converts special characters. We will discuss these functions when we cover ASVS 6.x (starting next post).

Example code putting it all together:

  $safeData = htmlentities(realpath($data), ENT_QUOTES, ISO-8859-1, false);

 
ASVS 5.9 Requirement:

Verify that all input validation controls are not affected by any malicious code.

ASVS 5.9 Solution:

Scan your code for viruses, or any malicious backdoors. Perform manual and automated code reviews.

We are finally starting a new OWASP Top 10 security risk today. The next few weeks (or possibly month) will cover XSS prevention techniques in PHP. This is probably a little more glamorous/sexy than authentication and session management. Controls to protect an application against XSS are primarily represented in the OWASP ASVS sections 5 and 6. Today, we start with ASVS section 5.

On to the code…

ASVS 5.1 Requirement:

Verify that the runtime environment is not susceptible to buffer overflows, or that security controls prevent buffer overflows.

ASVS 5.1 Solution:

This is more of a patch management issue than a coding problem in most cases (with PHP that is). Make sure your version of PHP is up to date. If using an older version, research potential vulnerabilities, perform a vulnerability scan, and make sure your code is not using any affected modules. However, I suggest using a newer and patched version. Validate that there are no vulnerabilities in your web server software, any frameworks that have been deployed, etc.

ASVS 5.2 Requirement:

Verify that a positive validation pattern is defined and applied to all input.

ASVS 5.2 Solution:

This requires a whitelist validation approach where only known good characters are accepted. Limiting the acceptable input to a specific set of characters reduces the risk of XSS, other forms of injection attacks (SQL injection, LDAP injection, etc), and several other classes of attack. It is also important to decode the input data before performing any validation. Decoding the data and performing canonicalization will help prevent filter bypass attacks that rely on encoding the data in a different character set. This can be implemented fairly easily with a PHP function like so:

  // Create a function that takes in the type of data
  // and the actual data itself.  The data type will 
  // be used to determine what kind of regex to use.
  function dataValidator($type, $data) {

    // Decode the data into a standard character
    // set before performing any checks
    $decodeData = html_entity_decode($data, ENT_QUOTES, 'UTF-8');

    // Declare an array that defines each data type 
    // name along with the regex.  In this example,
    // we are only defining three types to accept: 
    // number - only numbers in the data, letter - 
    // only alphabet characters in the data, and 
    // alphan - alphanumeric characters in the data.
    $typeArray = array(
      "number" => '/^[0-9]+$/',
      "letter" => '/^[a-zA-Z]+$/',
      "alphan" => '/^[a-zA-Z0-9]+$/'
    );

    // Use the PHP preg_match function to determine 
    // if the data only contains those characters.
    // Select the regex type by passing the "type" 
    // submitted to the function in to the array.
    // If there is a match, then the data is good
    // and the if statement returns true, otherwise
    // return false.
    if (preg_match($typeArray[$type], $decodeData)) {
      return true;
    } else {
      return false;
    }
  }

 
The above function could be implemented in a file that is included in any page that validates user supplied data. The function could be called with dataValidator(“number”, $_POST[‘userdata’]) (as an example) and the return value evaluated to determine whether bad data was submitted.

Whitelist validation is ideal, but sometimes blacklist validation is required or preferable. You can use similar code for a blacklist, just modify the regex and return true if the regex doesn’t match return values:

  // In this example, the regex's have been modified to
  // include everything that SHOULD NOT be found in the 
  // data type.  So a number value shouldn't contain 
  // letters from the alphabet or special characters.
  function dataValidator($type, $data) {
    $decodeData = html_entity_decode($data, ENT_QUOTES, 'UTF-8');

    $typeArray = array(
      "number" => '/^[a-zA-Z!\*,\'\"\|\.\\/\?\@\$\%\:\+\\(\)\[\]\{\}]+$/',
      "letter" => '/^[0-9!\*,\'\"\|\.\\/\?\@\$\%\:\+\\(\)\[\]\{\}]+$/',
      "alphan" => '/^[!\*\'\"\|\\/\?\@\$\%\+\\(\)\[\]\{\}]+$/'
    );

    // Here we return true if the regex DOESN'T match.
    if (!preg_match($typeArray[$type], $decodeData)) {
      return true;
    } else {
      return false;
    }
  }

ASVS 5.3 Requirement:

Verify that all input validation failures result in input rejection or input sanitization.

ASVS 5.3 Solution:

Implement a generic error message that is displayed if the return type from dataValidator() is false and reject the input. Alternatively, remove all non-matching characters. You can modify the code above to sanitize user supplied data like so:

  function dataValidator($type, $data) {
    $decodeData = html_entity_decode($data, ENT_QUOTES, 'UTF-8');

    $typeArray = array(
      "number" => '/^[0-9]+$/',
      "letter" => '/^[a-zA-Z]+$/',
      "alphan" => '/^[a-zA-Z0-9]+$/'
    );

    // Implement a second array for sanitizing.  Anything that
    // doesn't match will be removed later in the code.
    $sanitizeArray = array(
      "number" => '/[^0-9]/',
      "letter" => '/[^a-zA-Z]/',
      "alphan" => '/[^a-zA-Z0-9]/'
    );

    if (preg_match($typeArray[$type], $decodeData)) {
      // Return the input data because it passed the validation.
      return $decodeData;
    } else {
      // The validation failed, therefore some bad data was 
      // passed that needs to be removed.  Then return the
      // sanitized data.
      $sanitized = preg_replace($sanitizeArray[$type], '', $decodeData);
      return $sanitized;
    }
  }

ASVS 5.4 Requirement:

Verify that a character set, such as UTF-8, is specified for all sources of input.

ASVS 5.4 Solution:

This can be accomplished by forcing the data type. If an attacker can force a specific character set, then they might be able to bypass your data validation filters. An example of forcing UTF-8, and silently ignore/drop any characters that can’t be represented by that character set:

  // Supply the user supplied data to the forceUTF function.
  // This function will use the iconv PHP function to convert
  // the data to UTF-8, and drop any characters that can't
  // be converted into the charset.
  function forceUTF($data) {
    $utfEncoded = iconv("UTF-8", "UTF-8//IGNORE", $data);
    return $utfEncoded;
  }

ASVS 5.5 Requirement:

Verify that all input validation is performed on the server side.

ASVS 5.5 Solution:

Do not perform input validation on the client side as this is trivial to bypass and gives an attacker clues as to how data might be validated on the server side.

I think this is a good place to leave off for the week. Please provide feedback if you have it.