Knowledge base for a port authority
Knowledge base for a port authority
A document collector and an artificial intelligence service that answers with sources, for a port authority.
The problem
People working at a port authority have to follow regulations, documents and environmental news published on many different institutional websites. The information is there, but scattered: finding and checking it takes time.
They needed one place to search, with answers whose origin can be checked.
What I built
- 01
The collector
A crawler that visits the pages and downloads the documents of the indicated institutional websites, and keeps them up to date.
- 02
The index
An artificial intelligence service that indexes texts and documents.
- 03
Answers with sources
Whoever asks a question gets an answer that points to the passages and documents it comes from.
- 04
The browsing website
A site to browse and search, designed for people working at the authority.
My role
Developer at ETT on the project. I worked on the page and document collector, on the service that indexes them and on the website. The client is not named.
Technologies
Python · Django · crawler · AI for search · Docker
With institutional documents the hard part is collecting well, not answering well: different formats, pages that change, duplicates. Also, an answer without a source is of no use to someone who then has to sign an official act.
Documents spread over several sites
Tell me what you need to find and for whom, and we will assess together whether a knowledge base is needed.
Stock photo: “Container cranes at the MPET – MSC PSA European Terminal in Port of Antwerp” by Trougnouf, licence CC BY 4.0 (Wikimedia Commons). It does not show the project.