How VFD-Checker Works
Once a week, every company listed in the Vertical Farming Directory is checked automatically. The results determine the status shown on each company's directory profile.
The crawl process
-
Fetch all active listings
The checker retrieves the current list of all companies from the Vertical Farming Directory — name, website URL, and category.
-
Check each domain
Each company's homepage is requested. We record whether the page loads, how quickly it responds, and whether it redirects to another location.
-
Detect structured data
If the page loads successfully, we check whether it contains any structured data signals — a VFD meta tag, Schema.org Organisation markup, or Open Graph tags. These help us confirm the site is actively maintained, not just reachable.
-
Classify the result
Each domain receives a status: Active, Redirect, Unreachable, or Error. Domains that fail repeatedly over several weeks are escalated for review.
-
Update directory profiles
Results flow into the Vertical Farming Directory. Company profiles reflect the latest status — active companies with a verified signal receive a badge on their profile; repeatedly unreachable domains are flagged or deactivated.
Status definitions
Domain responds normally
The website loads and returns a valid response. The company is considered online.
Domain redirects elsewhere
The domain redirects to another address. If the destination is a domain parking or auction service, the domain is treated as inactive.
Domain does not respond
The domain cannot be reached, returns an error, or has been taken over by a parking page. After three consecutive failed checks, the listing is flagged for manual review.
Temporary issue
The request timed out or the server returned a temporary error. A single error does not flag the domain — only repeated failures over multiple weeks trigger a review.
What we do not do
- We do not crawl subpages or follow links beyond the homepage
- We do not store page content — only status signals and response metadata
- We do not submit forms or interact with JavaScript
- We respect
robots.txt— domains that opt out are marked as crawl-blocked, not inactive