Favicons (source)¶
Implementations for providing the favicons in SearXNG
- searx.favicons.favicon_proxy()[source]¶
REST API of SearXNG’s favicon proxy service
/favicon_proxy?authority=<...>&h=<...>
authority
:Domain name RFC 3986 / see
favicon_url
h
:HMAC RFC 2104, build up from the server.secret_key setting.
- searx.favicons.favicon_url(authority: str) str [source]¶
Function to generate the image URL used for favicons in SearXNG’s result lists. The
authority
argument (aka netloc / RFC 3986) is usually a (sub-) domain name. This function is used in the HTML (jinja) templates.<div class="favicon"> <img src="{{ favicon_url(result.parsed_url.netloc) }}"> </div>
The returned URL is a route to
favicon_proxy
REST API.If the favicon is already in the cache, the returned URL is a data URL (something like
data:image/png;base64,...
). By generating a data url from thecache.FaviconCache
, additional HTTP roundtripps via thefavicon_proxy
are saved. However, it must also be borne in mind that data urls are not cached in the client (web browser).
Favicons Config¶
- class searx.favicons.config.FaviconConfig(cfg_schema: int, cache: ~searx.favicons.cache.FaviconCacheConfig = <factory>, proxy: ~searx.favicons.proxy.FaviconProxyConfig = <factory>)[source]¶
The class aggregates configurations of the favicon tools
- classmethod from_toml_file(cfg_file: Path, use_cache: bool) FaviconConfig [source]¶
Create a config object from a TOML file, the
use_cache
argument specifies whether a cache should be used.
- cache: FaviconCacheConfig¶
Setup of the
cache.FaviconCacheConfig
.
- cfg_schema: int¶
Config’s schema version. The specification of the version of the schema is mandatory, currently only version
CONFIG_SCHEMA
is supported. By specifying a version, it is possible to ensure downward compatibility in the event of future changes to the configuration schema
- proxy: FaviconProxyConfig¶
Setup of the
proxy.FaviconProxyConfig
.
- searx.favicons.config.TOML_CACHE_CFG: dict[str, FaviconConfig] = {}¶
Cache config objects by TOML’s filename.
Favicons Proxy¶
Implementations for a favicon proxy
- class searx.favicons.proxy.FaviconProxyConfig(max_age: int = 604800, secret_key: str = '', resolver_timeout: int = 3.0, resolver_map: dict[str, str] = <factory>, favicon_path: str = '/home/runner/work/searxng/searxng/searx/static/themes/{theme}/img/empty_favicon.svg', favicon_mime_type: str = 'image/svg+xml')[source]¶
Configuration of the favicon proxy.
- get_resolver(name: str) Callable | None [source]¶
Returns the callable object (function) of the resolver with the
name
. If no resolver is registered for thename
,None
is returned.
- max_age: int¶
HTTP header Cache-Control
max-age
- resolver_map: dict[str, str]¶
The resolver_map is a key / value dictionary where the key is the name of the resolver and the value is the fully qualifying name (fqn) of resolver’s function (the callable). The resolvers from the python module
searx.favicons.resolver
are available by default.
- resolver_timeout: int¶
Timeout which the resolvers should not exceed, is usually passed to the outgoing request of the resolver. By default, the value from outgoing.request_timeout setting is used.
- secret_key: str¶
By default, the value from server.secret_key setting is used.
- searx.favicons.proxy.favicon_proxy()[source]¶
REST API of SearXNG’s favicon proxy service
/favicon_proxy?authority=<...>&h=<...>
authority
:Domain name RFC 3986 / see
favicon_url
h
:HMAC RFC 2104, build up from the server.secret_key setting.
- searx.favicons.proxy.favicon_url(authority: str) str [source]¶
Function to generate the image URL used for favicons in SearXNG’s result lists. The
authority
argument (aka netloc / RFC 3986) is usually a (sub-) domain name. This function is used in the HTML (jinja) templates.<div class="favicon"> <img src="{{ favicon_url(result.parsed_url.netloc) }}"> </div>
The returned URL is a route to
favicon_proxy
REST API.If the favicon is already in the cache, the returned URL is a data URL (something like
data:image/png;base64,...
). By generating a data url from thecache.FaviconCache
, additional HTTP roundtripps via thefavicon_proxy
are saved. However, it must also be borne in mind that data urls are not cached in the client (web browser).
- searx.favicons.proxy.search_favicon(resolver: str, authority: str) tuple[None | bytes, None | str] [source]¶
Sends the request to the favicon resolver and returns a tuple for the favicon. The tuple consists of
(data, mime)
, if the resolver has not determined a favicon, both values areNone
.data
:Binary data of the favicon.
mime
:Mime type of the favicon.
Favicons Resolver¶
Implementations of the favicon resolvers that are available in the favicon
proxy by default. A resolver is a function that obtains the favicon from an
external source. The resolver function receives two arguments (domain,
timeout
) and returns a tuple (data, mime)
.
- searx.favicons.resolvers.allesedv(domain: str, timeout: int) tuple[None | bytes, None | str] [source]¶
Favicon Resolver from allesedv.com / https://favicon.allesedv.com/
- searx.favicons.resolvers.duckduckgo(domain: str, timeout: int) tuple[None | bytes, None | str] [source]¶
Favicon Resolver from duckduckgo.com / https://blog.jim-nielsen.com/2021/displaying-favicons-for-any-domain/
Favicons Cache¶
Implementations for caching favicons.
FaviconCacheConfig
:Configuration of the favicon cache
FaviconCache
:Abstract base class for the implementation of a favicon cache.
FaviconCacheSQLite
:Favicon cache that manages the favicon BLOBs in a SQLite DB.
FaviconCacheNull
:Fallback solution if the configured cache cannot be used for system reasons.
- class searx.favicons.cache.FaviconCache(cfg: FaviconCacheConfig)[source]¶
Abstract base class for the implementation of a favicon cache.
- abstract set(resolver: str, authority: str, mime: str | None, data: bytes | None) bool [source]¶
Set data and mime-type in the cache. If data is None, the
FALLBACK_ICON
is registered. in the cache.
- abstract state() FaviconCacheStats [source]¶
Returns a
FaviconCacheStats
(key/values) with information on the state of the cache.
- class searx.favicons.cache.FaviconCacheConfig(db_type: Literal['sqlite', 'mem'] = 'sqlite', db_url: str = '/tmp/faviconcache.db', HOLD_TIME: int = 2592000, LIMIT_TOTAL_BYTES: int = 52428800, BLOB_MAX_BYTES: int = 20480, MAINTENANCE_PERIOD: int = 3600, MAINTENANCE_MODE: Literal['auto', 'off'] = 'auto')[source]¶
Configuration of the favicon cache.
- BLOB_MAX_BYTES: int¶
The maximum BLOB size in bytes that a favicon may have so that it can be saved in the cache. If the favicon is larger, it is not saved in the cache and must be requested by the client via the proxy.
- LIMIT_TOTAL_BYTES: int¶
Maximum of bytes (default) stored in the cache of all blobs. Note: The limit is only reached at each maintenance interval after which the oldest BLOBs are deleted; the limit is exceeded during the maintenance period. If the maintenance period is too long or maintenance is switched off completely, the cache grows uncontrollably.
- MAINTENANCE_MODE: Literal['auto', 'off']¶
Type of maintenance mode
auto
:Maintenance is carried out automatically as part of the maintenance intervals (
MAINTENANCE_PERIOD
); no external process is required.off
:Maintenance is switched off and must be carried out by an external process if required.
- MAINTENANCE_PERIOD: int¶
Maintenance period in seconds / when
MAINTENANCE_MODE
is set toauto
.
- db_type: Literal['sqlite', 'mem']¶
Type of the database:
sqlite
:mem
:cache.FaviconCacheMEM
(not recommended)
- class searx.favicons.cache.FaviconCacheMEM(cfg)[source]¶
Favicon cache in process’ memory. Its just a POC that stores the favicons in the memory of the process.
Attention
Don’t use it in production, it will blow up your memory!!
- set(resolver: str, authority: str, mime: str | None, data: bytes | None) bool [source]¶
Set data and mime-type in the cache. If data is None, the
FALLBACK_ICON
is registered. in the cache.
- state()[source]¶
Returns a
FaviconCacheStats
(key/values) with information on the state of the cache.
- class searx.favicons.cache.FaviconCacheNull(cfg: FaviconCacheConfig)[source]¶
A dummy favicon cache that caches nothing / a fallback solution. The NullCache is used when more efficient caches such as the
FaviconCacheSQLite
cannot be used because, for example, the SQLite library is only available in an old version and does not meet the requirements.- set(resolver: str, authority: str, mime: str | None, data: bytes | None) bool [source]¶
Set data and mime-type in the cache. If data is None, the
FALLBACK_ICON
is registered. in the cache.
- state()[source]¶
Returns a
FaviconCacheStats
(key/values) with information on the state of the cache.
- class searx.favicons.cache.FaviconCacheSQLite(cfg: FaviconCacheConfig)[source]¶
Favicon cache that manages the favicon BLOBs in a SQLite DB. The DB model in the SQLite DB is implemented using the abstract class
sqlitedb.SQLiteAppl
.The following configurations are required / supported:
MAINTENANCE_PERIOD
MAINTENANCE_MODE
- set(resolver: str, authority: str, mime: str | None, data: bytes | None) bool [source]¶
Set data and mime-type in the cache. If data is None, the
FALLBACK_ICON
is registered. in the cache.
- state() FaviconCacheStats [source]¶
Returns a
FaviconCacheStats
(key/values) with information on the state of the cache.
- DB_SCHEMA: int = 1¶
As soon as changes are made to the DB schema, the version number must be increased. Changes to the version number require the DB to be recreated (or migrated / if an migration path exists and is implemented).
- DDL_BLOBS = 'CREATE TABLE IF NOT EXISTS blobs (\n sha256 TEXT,\n bytes_c INTEGER,\n mime TEXT NOT NULL,\n data BLOB NOT NULL,\n PRIMARY KEY (sha256))'¶
Table to store BLOB objects by their sha256 hash values.
- DDL_BLOB_MAP = "CREATE TABLE IF NOT EXISTS blob_map (\n m_time INTEGER DEFAULT (strftime('%s', 'now')), -- last modified (unix epoch) time in sec.\n sha256 TEXT,\n resolver TEXT,\n authority TEXT,\n PRIMARY KEY (resolver, authority))"¶
Table to map from (resolver, authority) to sha256 hash values.
- SQL_DROP_LEFTOVER_BLOBS = 'DELETE FROM blobs WHERE sha256 IN ( SELECT b.sha256 FROM blobs b LEFT JOIN blob_map bm ON b.sha256 = bm.sha256 WHERE bm.sha256 IS NULL)'¶
Delete blobs.sha256 (BLOBs) no longer in blob_map.sha256.
- class searx.favicons.cache.FaviconCacheStats(favicons: int | None = None, bytes: int | None = None, domains: int | None = None, resolvers: int | None = None)[source]¶
Dataclass wich provides information on the status of the cache.
- searx.favicons.cache.init(cfg: FaviconCacheConfig)[source]¶
Initialization of a global
CACHE