Page MenuHomeVyOS Platform

webproxy squidguard rules don't work properly after rewriting to python.
Closed, ResolvedPublicBUG

Description

1. Missed part of the squidguard configuration:

run update webproxy blacklists

set service webproxy listen-address 192.168.122.15 disable-transparent
set service webproxy listen-address 192.168.122.15 port '3128'
set service webproxy url-filtering squidguard default-action 'block'
set service webproxy url-filtering squidguard rule 1 block-category 'social_networks'
set service webproxy url-filtering squidguard rule 1 source-group social
set service webproxy url-filtering squidguard source-group social address '192.168.122.0/24'

Get configuration:

[email protected]# sudo cat /etc/squidguard/squidGuard.conf
### generated by service_webproxy.py ###


dbhome /opt/vyatta/etc/config/url-filtering/squidguard/db
logdir /var/log/squid

rewrite safesearch {
        s@(.*\.google\..*/(custom|search|images|groups|news)?.*q=.*)@\1\&safe=active@i
        s@(.*\..*/yandsearch?.*text=.*)@\1\&fyandex=1@i
        s@(.*\.yahoo\..*/search.*p=.*)@\1\&vm=r@i
        s@(.*\.live\..*/.*q=.*)@\1\&adlt=strict@i
        s@(.*\.msn\..*/.*q=.*)@\1\&adlt=strict@i
        s@(.*\.bing\..*/search.*q=.*)@\1\&adlt=strict@i
        log     rewrite.log
}


acl {
    default {
        pass local-ok-default !in-addr none
        redirect 302:http://block.vyos.net
    }
}

Expected configuration:

vyos@r12-lts# sudo cat /etc/squidguard/squidGuard.conf
#
# autogenerated by vyatta-update-webproxy.pl
#

dbhome /opt/vyatta/etc/config/url-filtering/squidguard/db
logdir /var/log/squid

rewrite safesearch {
	s@(.*\.google\..*/(custom|search|images|groups|news)?.*q=.*)@\1\&safe=active@i
	s@(.*\..*/yandsearch?.*text=.*)@\1\&fyandex=1@i
	s@(.*\.yahoo\..*/search.*p=.*)@\1\&vm=r@i
	s@(.*\.live\..*/.*q=.*)@\1\&adlt=strict@i
	s@(.*\.msn\..*/.*q=.*)@\1\&adlt=strict@i
	s@(.*\.bing\..*/search.*q=.*)@\1\&adlt=strict@i
	log	rewrite.log
}

src social-1 {
	ip 192.168.122.0/24
}
dest local-ok-default {
	domainlist     local-ok-default/domains
}

dest local-ok-url-default {
	urllist        local-ok-url-default/urls
}

dest local-ok-1 {
	domainlist     local-ok-1/domains
}

dest local-ok-url-1 {
	urllist        local-ok-url-1/urls
}

dest social_networks-1 {
	domainlist     social_networks/domains
	urllist        social_networks/urls
}

acl {
	social-1 {
		pass local-ok-1 !in-addr !social_networks-1 all
	}

	default {
		pass local-ok-default !in-addr none
		redirect 302:http://block.vyos.net
	}

}

2. ̶b̶u̶g̶ ̶p̶e̶r̶m̶i̶s̶s̶i̶o̶n̶ ̶e̶r̶r̶o̶r̶

done

vyos@r1-roll# set service webproxy url-filtering squidguard rule 1 block-category 'social_networks'
ls: cannot access '/opt/vyatta/etc/config/url-filtering/squidguard/db//*': Permission denied

3. N̶o̶d̶e̶ ̶a̶d̶d̶r̶e̶s̶s̶ ̶s̶h̶o̶u̶l̶d̶ ̶b̶e̶ ̶/̶m̶u̶l̶t̶i̶

done

set service webproxy url-filtering squidguard source-group social address 192.0.2.0/24
set service webproxy url-filtering squidguard source-group social address 203.0.113.0/24

https://github.com/vyos/vyos-1x/blob/adca504a2c5cd60be46a741ab3aef83fa4dfe4cf/interface-definitions/service_webproxy.xml.in#L496-L517

4. ̶T̶h̶e̶r̶e̶ ̶i̶s̶ ̶n̶o̶ ̶"̶s̶o̶u̶r̶c̶e̶-̶g̶r̶o̶u̶p̶"̶ ̶i̶n̶ ̶t̶e̶m̶p̶l̶a̶t̶e̶ ̶

done

set service webproxy url-filtering squidguard source-group

https://github.com/vyos/vyos-1x/blob/current/data/templates/squid/squidGuard.conf.tmpl

5. No any options for "rule options" in template

vyos@r1-roll# set service webproxy url-filtering squidguard rule 1 
Possible completions:
+  allow-category
                Category to allow
   allow-ipaddr-url
                Allow IP address URLs
+  block-category
                Category to block
   default-action
                Default action (default: allow)
   enable-safe-search
                Enable safe-mode search on popular search engines
+  local-block  Local site to block
+  local-block-keyword
                Local keyword to block
+  local-block-url
                Local URL to block
+  local-ok     Local site to allow
+  local-ok-url Local URL to allow
+  log          Log block category
   redirect-url Redirect URL for filtered websites
   source-group Source-group for this rule [REQUIRED]
   time-period  Time-period for this rule

6 ̶"̶a̶c̶l̶ ̶l̶o̶c̶a̶l̶h̶o̶s̶t̶"̶ ̶a̶n̶d̶ ̶"̶a̶c̶l̶ ̶t̶o̶_̶l̶o̶c̶a̶l̶h̶o̶s̶t̶"̶ ̶g̶e̶n̶e̶r̶a̶t̶e̶d̶ ̶i̶n̶ ̶s̶q̶u̶i̶d̶ ̶b̶y̶ ̶d̶e̶f̶a̶u̶l̶t̶ ̶(̶b̶u̶i̶l̶t̶i̶n̶ ̶t̶o̶ ̶s̶q̶u̶i̶d̶3̶)̶

done
So we don't need to declare it again in the template http://www.squid-cache.org/Versions/v3/3.2/cfgman/acl.html

Sep  9 11:45:33 r1-roll (squid-1): WARNING: (B) '127.0.0.1' is a subnetwork of (A) '127.0.0.1'
Sep  9 11:45:33 r1-roll (squid-1): WARNING: because of this '127.0.0.1' is ignored to keep splay tree searching predictable
Sep  9 11:45:33 r1-roll (squid-1): WARNING: You should probably remove '127.0.0.1' from the ACL named 'localhost'
Sep  9 11:45:33 r1-roll (squid-1): WARNING: (B) '127.0.0.1' is a subnetwork of (A) '127.0.0.1'
Sep  9 11:45:33 r1-roll (squid-1): WARNING: because of this '127.0.0.1' is ignored to keep splay tree searching predictable
Sep  9 11:45:33 r1-roll (squid-1): WARNING: You should probably remove '127.0.0.1' from the ACL named 'localhost'
Sep  9 11:45:33 r1-roll (squid-1): WARNING: (B) '127.0.0.0/8' is a subnetwork of (A) '127.0.0.0/8'
Sep  9 11:45:33 r1-roll (squid-1): WARNING: because of this '127.0.0.0/8' is ignored to keep splay tree searching predictable
Sep  9 11:45:33 r1-roll (squid-1): WARNING: You should probably remove '127.0.0.0/8' from the ACL named 'to_localhost'
Sep  9 11:45:33 r1-roll (squid-1): WARNING: (B) '127.0.0.0/8' is a subnetwork of (A) '127.0.0.0/8'
Sep  9 11:45:33 r1-roll (squid-1): WARNING: because of this '127.0.0.0/8' is ignored to keep splay tree searching predictable
Sep  9 11:45:33 r1-roll (squid-1): WARNING: You should probably remove '127.0.0.0/8' from the ACL named 'to_localhost'

https://github.com/vyos/vyos-1x/blob/4d2201eed00ac4780d0196abf53dd9b7cb943a09/data/templates/squid/squid.conf.tmpl#L3-L4

7. ̶O̶l̶d̶ ̶d̶i̶r̶e̶c̶t̶i̶v̶e̶ ̶"̶r̶e̶d̶i̶r̶e̶c̶t̶_̶p̶r̶o̶g̶r̶a̶m̶"̶

done
url_rewrite_program replaces redirect_program
url_rewrite_children replaces redirect_children
url_rewrite_bypass replaces redirector_bypass
http://www.squid-cache.org/Doc/config/url_rewrite_program/
https://github.com/vyos/vyos-1x/blob/310eb1b527047211ae236c6415fee51f15a0fa57/data/templates/squid/squid.conf.tmpl#L104

8. Files not exists in db

The files "local-ok-1" and "local-ok-default" not exist in /opt/vyatta/etc/config/url-filtering/squidguard/db
Needs to figure out if we need it or delete from template. Otherwise filters may not work.

Details

Difficulty level
Normal (likely a few hours)
Version
VyOS 1.3.0-rc6, VyOS 1.4-rolling-202109061053
Why the issue appeared?
Will be filled on close
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Unspecified (please specify)

Event Timeline

Viacheslav renamed this task from webproxy squidguard rules doesn't work properly to webproxy squidguard rules don't work properly after rewriting to python. .Sep 9 2021, 3:10 PM
Viacheslav changed the task status from Open to Needs testing.Sep 13 2021, 7:56 AM
Viacheslav updated the task description. (Show Details)
a.apostoliuk changed the task status from Needs testing to In progress.Nov 29 2022, 7:46 AM
a.apostoliuk claimed this task.
c-po changed the task status from In progress to Backport candidate.Dec 17 2022, 7:21 AM
Viacheslav changed the task status from Backport candidate to Needs testing.Jan 9 2023, 1:23 PM
a.apostoliuk moved this task from Need Triage to Finished on the VyOS 1.4 Sagitta board.