Support Support Downloads Knowledge Base Case Manager My Juniper Community

Knowledge Base

Search our Knowledge Base sites to find answers to your questions.

Ask All Knowledge Base Sites All Knowledge Base Sites JunosE Defect (KA)Knowledge BaseSecurity AdvisoriesTechnical BulletinsTechnotes Sign in to display secure content and recently viewed articles

[SRX] Integrated web filtering example via custom objects.

0

0

Article ID: KB17287 KB Last Updated: 22 Feb 2020Version: 3.0
Summary:

This article provides information on how to configure integrated Web filtering by using a custom URL pattern and URL category lists from a category server (SurfControl Content Portal Authority provided by Websense).

Symptoms:
  • This article provides an example of configuring integrated Web filtering (also known as URL filtering) by using custom objects.

  • For basic information, additional examples, and troubleshooting about integrated Web filtering, refer to KB16334 - SRX Getting Started - Integrated Web Filtering.
Solution:

This section contains the following sections:

 

Configuration Task Overview

Configuring integrated Web filtering consists of the following tasks:

  • Configuring UTM custom objects and assigning them to categories 

  • Configuring integrated Web filtering parameters

  • Configuring a UTM policy for each protocol and attaching the policy to a profile

  • Attaching the UTM policy to a firewall security policy
 

Note: Integrated Web Filtering requires a license.  Run the show system license command, and look for wf_key_surfcontrol_cpa.


J-Web Configuration

To configure the integrated Web filtering feature profile:

  1. Go to Configure>Security>UTM>Global options and click the Web Filtering tab.

  2. from the Filtering Type list, select Surf Control Integrated.

  3. Click OK. A status prompt is displayed; click OK. If the custom object is not successfully saved, click Details for more information.
 

To configure a UTM policy for Web filtering:

  1. Go to Configure>Security>Policy>UTM Policies and click Add to configure a UTM policy; the Add Policy window is displayed.

  2. In the Main tab, next to Policy Name, type a unique name for the UTM policy (for example, custom-utm-policy).

  3. Click the Web filtering profiles tab.

  4. Next to the HTTP profile, select junos-wf-cpa-default and click OK.

  5. A status prompt is displayed; click OK. If the UTM policy is not successfully saved, click Details for more information.
 

To attach the UTM policy to a security policy:

  1. Go to Configure>Security>Policy>FW Policies.

  2. Select the trust-to-untrust (default-permit) security policy and click Edit.

  3. In the Edit Policy window, click Application Services.

  4. In the UTM Policy list, select the required UTM policy to attach to the security policy (in this example, custom-utm-policy).

  5. Click OK. A status prompt is displayed; click OK. If the UTM policy is not successfully saved, click Details for more information.

Note: Make sure that the policy is activated. By default, after a policy is created, it is activated.


To create an URL pattern list custom object:

  1. Go to Configure>Security>UTM>Custom Objects and click the URL Pattern List tab.

  2. Click Add to create URL pattern lists; the Add URL Pattern window is displayed.

  3. Next to URL Pattern Name, type a unique name for the list (for example, black-list or white-list).

  4. Next to URL Pattern Value, type the URL or IP address that has to be added to the list (for example, http://*.hacking.com).

  5. Click Add to add the URL pattern; the pattern is displayed in the the Values list.

  6. To add more URLs or IP addresses, repeat steps 5 and 6.

  7. Click OK to save the URL pattern list.

  8. A status prompt is displayed; click OK. If the URL pattern list is not successfully saved, click Details for more information.
 

To create a custom URL category list custom object:

  1. Go to Configure>Security>UTM>Custom Objects and click the URL Category List tab.

  2. Click Add to create URL category lists; the Add URL Category window is displayed.

  3. Next to URL Category Name, type a unique name for the URL category list custom object (for example, blocked-sites).

  4. From the Available Values list, select the previously created URL pattern (for example, black-list from the previous procedure) and click the right arrow button to move it to the Selected Values list.

  5. To add more values, repeat step 5.

  6. Click OK. A status prompt is displayed; click OK. If the URL category list is not successfully saved, click Details for more information.
 

To configure the redirect Web filtering feature profile:

  1. Go to Configure>Security>UTM>Global options and click the Web Filtering tab.

  2. Next to URL whitelist, select the previously created URL category (in this example, allowed-sites from the previous procedure).

  3. Next to URL blacklist, select the previously created URL category (in this example, blocked-sites from the previous procedure).

  4. From the Filtering Type list, select Surf Control Integrated.

  5. Next to Cache timeout, type the timeout (in minutes) for expiring cache entries (for example, values 1- 1800).

  6. Next to Cache Size, type the maximum number of kilobytes (KB) for the cache (for example, 500).

  7. Next to Server Host, type the Surf Control server name or IP address (for example, cpa.surfcpa.com).

  8. Next to Server Port, type the port number used to communicate with the Surf Control server (for example, 9020).

  9. Click OK. A status prompt is displayed; click OK. If the custom object is not successfully saved, click Details for more information.

  10. In the left pane, under Security >UTM, click Web Filtering.

  11. Click Add to create a profile for integrated Web filtering.

  12. In the Main tab, next to Profile name, type a unique name for the Web filtering profile (for example, surfcontrol-profile1).

  13. In the Profile Type list, select Surf Control.

  14. Next to Default action, select either Permit, Log and permit, or Block (in this example, Block).

  15. Next to Timeout, type the timeout value (in seconds), at which fallback options are applied (for example, 20).

  16. Next to Custom Block Message, type the message that is sent when HTTP requests are blocked (for example, ***DENIED***).

  17. Click the Fallback options tab.

  18. Next to Default Action, select either Log and permit or Block as the action to be taken, when a request fails; due to it not matching any categories (in this example, Block).

  19. Next to Server Connectivity, either select Log and permit or Block as the action to be taken, when a request fails for this reason (in this example, Block).

  20. Next to Timeout, either select Log and permit or Block as the action to be taken, when a request fails for this reason (in this example, Block).

  21. Next to Too Many Requests, either select Log and permit or Block as the action to be taken, when a request fails for this reason (in this example, Block).

  22. Click the URL category action list tab.

  23. Next to Categories, select a configured custom object  (in this example, blocked-sites).

  24. Next to Actions, either select Permit, Block or Log and Permit (in this example, Block).

  25. Click Add to use a configured custom URL category list custom object in the profile.

  26. Click OK. A status prompt is displayed; click OK. If the Web filtering options are not successfully saved, click Details for more information.

To configure a UTM policy for Web filtering:
 
  1. Go to Configure>Security>Policy>UTM Policies and click Add to configure a UTM policy; the Add Policy window is displayed.

  2. Click the Main tab.

  3. In the Policy Name box, type a unique name for the UTM policy (for example, web-filter).

  4. In the Session per client limit box, type a session per client limit from 0 to 20000 for this UTM policy.

  5. For Session per client over limit, either select Log and Permit or Block. This is the action that the device takes when the session per client limit for this UTM policy is exceeded.

  6. Click the Web filtering profiles tab.

  7. Next to HTTP profile, select the previously configured profile (in this example, surfcontrol-profile1).

  8. Click OK. A status prompt is displayed; click OK. If the UTM policy is not successfully saved, click Details for more information.
 

To attach the UTM policy to a security policy:

  1. Go to Configure>Security>Policy>FW Policies and click Add; the Add Policy window is displayed.

  2. Click the Policy tab.

  3. In the Policy Name box, type the name of the policy (for example, intwebfilter).

  4. Next to From Zone, select a zone from the list (for example, trust).

  5. Next to To Zone, select a zone from the list (for example, untrust).

  6. Select a source address (for example, any).

  7. Select a destination address (for example, any).

  8. Select an application by selecting junos-http in the Application Sets box and click the arrow button.

  9. Next to Default Policy Action, select permit.
     
  10. Click the Application Services tab.

  11. Next to UTM Policy, select the UTM policy to be attached to the security policy (in this example, web-filter).

  12. Click OK. A status prompt is displayed; click OK. If the policy is not successfully saved, click Details for more information.
Note: Make sure that the policy is activated. By default, after a policy is created, it is activated.

 

CLI Configuration


The following example activates integrated Web filtering.

  1. Configure the device to use the integrated Web filtering feature.
user@host# set security utm feature-profile web-filtering type surf-control-integrated
  1. Create a UTM policy and associate the "JUNOS-wf-cpa-default" profile to the policy.
user@host# set security utm utm-policy custom-utm-policy web-filtering http-profile JUNOS-wf-cpa-default
  1. Apply the UTM policy to the existing trust-to-untrust security policy.
user@host# set security policies from-zone trust to-zone untrust policy default-permit then permit application-services utm-policy custom-utm-policy


To configure integrated Web filtering, create the UTM custom objects first. Custom objects are global parameters for UTM features and apply to all UTM policies where applicable, rather than only to individual policies. In this example, custom URL black and white lists are put into two separate categories.

  1. Define the custom URL pattern lists--black-list and white-list.

  2. user@host# set security utm custom-objects url-pattern black-list value http://*.sex.com
    user@host# set security utm custom-objects url-pattern black-list value http://*.guns.com
    user@host# set security utm custom-objects url-pattern black-list value http://*.hacking.com
    user@host# set security utm custom-objects url-pattern white-list value http://*.juniper.net
    user@host# set security utm custom-objects url-pattern white-list value http://*.cnn.net
    user@host#
    set security utm custom-objects url-pattern white-list value http://*.msn.net

  3. Define the custom URL categories allowed-sites and blocked-sites), by putting the white-list in one category and the black-list in the other category.

  4. user@host# set security utm custom-objects custom-url-category allowed-sites value white-list
    user@host# set security utm custom-objects custom-url-category blocked-sites value black-list

  5.  

After creating custom objects, configure the Web filtering feature parameters.

  1. Set the type of web-filtering to surf-control-integrated.

  2. user@host# set security utm feature-profile web-filtering type surf-control-integrated

  3. Define the global URL white and black lists.

  4. user@host# set security utm feature-profile web-filtering url-whitelist allowed-sites
    user@host#
    set security utm feature-profile web-filtering url-blacklist blocked-sites

  5. Define the SurfControl server settings.

  6. user@host# set security utm feature-profile web-filtering surf-control-integrated cache timeout 1800
    user@host# set security utm feature-profile web-filtering surf-control-integrated cache size 500
    user@host# set security utm feature-profile web-filtering surf-control-integrated server host cpa.surfcpa.com
    user@host#
    set security utm feature-profile web-filtering surf-control-integrated server port 9020

  7. Create the Web filtering profile and specify the actions to be taken for each category (user-defined and custom).

  8. user@host# set security utm feature-profile web-filtering surf-control-integrated profile surfcontrol-profile1 category Adult_Sexually_Explicit action block
    user@host# set security utm feature-profile web-filtering surf-control-integrated profile surfcontrol-profile1 category Hacking action block
    user@host# set security utm feature-profile web-filtering surf-control-integrated profile surfcontrol-profile1 category Weapons action block
    user@host# set security utm feature-profile web-filtering surf-control-integrated profile surfcontrol-profile1 category Web_based_Email action permit
    user@host# set security utm feature-profile web-filtering surf-control-integrated profile surfcontrol-profile1 default block
    user@host# set security utm feature-profile web-filtering surf-control-integrated profile surfcontrol-profile1 custom-block-message ***DENIED***

  9. Define the fallback settings for the Web filtering profile. The fallback options define the actions to be taken for traffic when errors in each configured category occur.

  10. user@host# set security utm feature-profile web-filtering surf-control-integrated profile surfcontrol-profile1 fallback-settings default block
    user@host# set security utm feature-profile web-filtering surf-control-integrated profile surfcontrol-profile1 fallback-settings server-connectivity block
    user@host# set security utm feature-profile web-filtering surf-control-integrated profile surfcontrol-profile1 fallback-settings timeout block
    user@host#
    set security utm feature-profile web-filtering surf-control-integrated profile surfcontrol-profile1 fallback-settings too-many-requests block

  11.  

Define the UTM policy for the protocol and attach this policy to a profile. Then apply the UTM policy to a firewall security policy as an application service.

  1. Define the UTM policy for HTTP (web-filter) and attach this policy to a profile (surfcontrol-profile1).

  2. user@host# set security utm utm-policy web-filter web-filtering http-profile surfcontrol-profile1

  3. Apply the UTM policy to a policy from the Trust zone to the Untrust zone, and set the application services to be allowed:

  4. user@host# set security policies from-zone trust to-zone untrust policy web-filter match source-address any
    user@host# set security policies from-zone trust to-zone untrust policy web-filter match destination-address any
    user@host# set security policies from-zone trust to-zone untrust policy web-filter match application any
    user@host#
    set security policies from-zone trust to-zone untrust policy web-filter then permit application-services utm-policy web-filter


Full Working Configuration Example

version 10.1R1.8;
system {
    host-name Starburst;
    root-authentication {
        encrypted-password "$ABC123"; ## SECRET-DATA
    }
    login {
        message "/**** Please reload /var/tmp/default.conf when you are done ****/ ";
        user lab {
            uid 2000;
            class superuser;
            authentication {
                encrypted-password "$ABC123"; ## SECRET-DATA
            }
        }
    }
    services {
        ftp;
        ssh;
        telnet;
        web-management {
            http {
                interface ge-0/0/0.0;
            }
            https {
                system-generated-certificate;
                interface ge-0/0/0.0;
            }
        }
    }
    syslog {
        user * {
            any emergency;
        }
        file messages {
            any any;
            authorization info;
        }
        file interactive-commands {
            interactive-commands any;
        }
    }
}
interfaces {
    ge-0/0/0 {
        unit 0 {
            family inet {
                address 10.10.66.94/24;
            }
        }
    }
}
routing-options {
    static {
        route 66.129.243.0/24 {
            next-hop 10.10.66.1;
            no-readvertise;
        }
    }
}
security {
    zones {
        security-zone trust {
            interfaces {
                ge-0/0/0.0 {
                    host-inbound-traffic {
                        system-services {
                            all;
                        }
                        protocols {
                            all;
                        }
                    }
                }
            }
        }
        security-zone untrust;
    }
    policies {
        from-zone trust to-zone untrust {
            policy web-filter {
                match {
                    source-address any;
                    destination-address any;
                    application any;
                }
                then {
                    permit {
                        application-services {
                            utm-policy web-filter;
                        }
                    }
                }
            }
        }
    }
    utm {
        custom-objects {
            url-pattern {
                black-list {
                    value [ http://*.sex.com http://*.guns.com http://*.hacking.com ];
                }
                white-list {
                    value [ http://*.juniper.net http://*.cnn.net http://*.msn.net ];
                }
            }
            custom-url-category {
                allowed-sites {
                    value white-list;
                }
                blocked-sites {
                    value black-list;
                }
            }
        }
        feature-profile {
            web-filtering {
                url-whitelist allowed-sites;
                url-blacklist blocked-sites;
                type surf-control-integrated;
                traceoptions {
                    flag all;
                }
                surf-control-integrated {
                    cache {
                        timeout 1800;
                        size 500;
                    }
                    server {
                        host surfcontrolserver;
                        port 8080;
                    }
                    profile surfcontrol-profile1 {
                        category {
                            Adult_Sexually_Explicit {
                                action block;
                            }
                            Hacking {
                                action block;
                            }
                            Weapons {
                                action block;
                            }
                            Web_based_Email {
                                action permit;
                            }
                        }
                        default block;
                        custom-block-message ***DENIED***;
                        fallback-settings {
                            default block;
                            server-connectivity block;
                            timeout block;
                            too-many-requests block;
                        }
                    }
                }
            }
        }
        utm-policy web-filter {
            web-filtering {
                http-profile surfcontrol-profile1;
            }
        }
    }
}
Comment on this article > Affected Products Browse the Knowledge Base for more articles related to these product categories. Select a category to begin.

Getting Up and Running with Junos

Getting Up and Running with Junos Security Alerts and Vulnerabilities Product Alerts and Software Release Notices Problem Report (PR) Search Tool EOL Notices and Bulletins JTAC User Guide Customer Care User Guide Pathfinder SRX High Availability Configurator SRX VPN Configurator Training Courses and Videos End User Licence Agreement Global Search