ResearchTheta Admin

Dashboard

Live system overview

—

Professors

—

Conferences

—

Active Jobs

—

Labs

—

Universities

Agent Status

View all →

Loading...

API Spend

$0.00

Today

$0.00

This Month

0% of budget Budget: $10

Validation Status

0 verified 0 unverified 0 pending

0% validated

System

HTML build —

Admin server :4000

Node.js —

Uptime —

Provider Health

Loading...

Data Freshness

Last file write · last crawler run

Loading...

Crawler Last Runs

Roadmap & Data Strategy

vs Competitors — What We're Missing

●

H-index on professor cards — Research.com ranks by citation impact; our publications field is often 0

●

Email alerts for deadlines & jobs — AcademicPositions sends job alerts; FindAPhD sends PhD project alerts

●

PhD project listings — FindAPhD shows advertised PhD projects under supervisors; we show supervisors but not open projects

●

Publication count / recent papers — CSRankings ranks by top-venue papers; our data has publications but mostly empty

●

Conference field breadth — We have 9 fields vs ACM SIGCHI's 30+; add Accessibility, Social Computing, CSCW subfields

●

"Recently added" badges — No platform shows what's new this week; easy differentiator

Data Update Strategies (Prioritised)

1

Weekly job crawl — Enable jobs_daily cron; FindAPhD + jobs.ac.uk block bots, add Handshake + LinkedIn jobs API

2

Semantic Scholar enrichment — Auto-fill publications + h-index from free Semantic Scholar API (key in .env)

3

Faculty URL enricher — Run enrich_faculty_urls.py monthly; fixes lab_url pointing to institute homepages

4

Conference date verification — Monthly scrape of official conf websites for deadline corrections (not just AI guessing)

5

Email digest — Weekly email: upcoming deadlines (30d), new jobs, new professors added — drives return visits

6

User-submitted corrections — "Report wrong info" button on each card → stores in GitHub issue or simple JSON

API Usage — Last 14 Days

AI Providers

Keys, budgets, usage

Routing mode: auto

#	Provider	Model	Key	Daily Budget	Today	Month
Loading...

Usage — Last 30 Days

Loading...

Task Routing

Assign AI provider per task — fallback chain auto-applies

Task	Description	Primary Provider	Fallback Chain	Est. Cost
Loading...

Fallback chain fires automatically when primary fails (quota/429). Groq is free (14,400 req/day Llama 3.3 70B) — set it before OpenAI to save money.

Professor Validation

AI-verify every professor entry, enrich missing fields

0

Verified

0

Partial

0

Unverified

0

Pending

Overall Progress

Idle

Processing: — 0 / 0

Run Validation

Country (blank = all)

Limit (0=all)

Resume

Only pending

Live Output

Live tail

Output will appear here when validation runs...

Manual Add — Professor

AI-powered

Give anything: a name, homepage URL, ORCID (0000-0002-…), or Google Scholar link. AI extracts the profile, enriches it (h-index, IDs, lab URL) and adds it with a manual-check flag.

Faculty Control

Crawl, enrich, and manage professor data

Idle

Auto-refresh

Professor Crawler

Gemini knowledge extraction — HCI/VR/Haptics faculty

Idle

Country / Region

Batch mode — run all countries

AI Provider

Scholar Enrichment

Semantic Scholar API — h-index, citations, paper count

Country filter

AI Provider

Rate limit: 100 req/5 min. Sleeps 90s on limit hit.

Quick Actions

Faculty Sources

Name	Type	Frequency	Last Run	Items	Status	Active
Click Refresh to load sources

Live Output

Run a crawler above to see output here...

Manual Add — Conference

AI-powered

Conference acronym + year, or its website URL.

Conference Control

Crawl CFP deadlines, dates, URLs, and committee data

Conference Crawler

Updates CFP deadlines, dates, and URLs (Gemini)

AI Provider

Committee Crawler

Scrapes TPC member lists for SIGCHI, IEEE VR, and related venues

AI Provider

Conference Sources

Name	Type	Frequency	Last Run	Items	Status	Active
Click Refresh to load sources

Live Output

Run a crawler above to see output here...

Manual Add — Job

AI-powered

Paste a job URL (LinkedIn, jobs.ac.uk, any career page). If the site blocks bots, paste the full job description text instead.

Jobs Control

Crawl academic jobs from multiple sources

Idle

Jobs Crawler

jobs.ac.uk, EURAXESS, Nature, FindAPhD, and more

Source

Add Job from URL

Scrape and add a single job posting directly by URL

Jobs Sources

Name	Type	Frequency	Last Run	Items	Status	Active
Click Refresh to load sources

Live Output

Run a crawler above to see output here...

Manual Add — Lab

AI-powered

Lab website URL works best — or lab name + university.

Labs Control

Crawl and manage research lab profiles

Labs Crawler

Discover and update HCI/VR/Haptics research labs worldwide

AI Provider

Currently 111 labs in database.

Labs Sources

Name	Type	Frequency	Last Run	Items	Status	Active
Click Refresh to load sources

Live Output

Run a crawler above to see output here...

Manual Add — University

AI-powered

University name or website URL.

University Control

Manage university and department data sources

University Crawler

Discover and update university and department profiles

AI Provider

University crawler coming soon.

Scrape Single URL

Test any faculty or university page directly

University Sources

Name	Type	Frequency	Last Run	Items	Status	Active
Click Refresh to load sources

Live Output

Run a crawler above to see output here...

Conference URL Check

Verify all conference website URLs are reachable

Progress

—

Results

0 OK 0 Error

Conference	URL	Status	Code
Click "Check All URLs" to start

Schedule

Auto-runs while admin-server.js is on

Schedules run only while node admin-server.js is running. Enable a job and leave admin server on overnight.

Crawler	Description	Schedule	Last Run	Status	Enabled
Loading...

Data Manager

View, search, and remove entries

—

Professors

—

Conferences

—

Jobs

—

Labs

Browse Professors

ID	Name	University	Country	Status	Links	Manual Check
Select filters and load

—

Page 1

SEO Pages

Review, approve and deploy static HTML pages for Google indexing

— total

— pending

— approved

— deployed

— rejected

	Name	University / Info	CC	Expertise	Status	Actions
Loading...

Deploy Control

Rebuild HTML and push to Hostinger

—

HTML Size

—

Admin Server

admin.researchtheta.com · PM2 (port 4000)

VPS: 187.77.159.108

Actions

1. Rebuild HTML

Assemble researchtheta.html from VPS source files + validate JS

2. Deploy to Hostinger

Upload HTML to researchtheta.com via SFTP

Export & Bake VPS Off-Switch

Export DB → bake into HTML → deploy. Run before turning off VPS.

Live Output

Click an action above to see output...

Logs Viewer

Session logs, crawl logs, API usage

Live

Select a log file above...

Backups

Automatic backup schedule & settings

VPS Backup (server)

Daily time (server UTC)

Keep last N daily backups

Backs up all data files to /var/backups/researchradar daily. Admin files weekly (Sundays).

PC Backup (your computer)

Daily time (PC local time)

Backup folder on PC

Keep last N backups

Your PC reads these settings each time it backs up and re-schedules itself. Changes apply on the next PC backup run.

VPS Backup Files

Loading...

SEO & Publish

One sitemap for everything: app + WordPress blogs & pages + future entity pages

Sitemap Status

Loading...

After the ONE-TIME submission of sitemap.xml in Google Search Console, Google re-fetches it automatically — clicking the button above is all you ever need when content changes. New WordPress posts are picked up automatically.

Request Indexing (single URL)

Sitemap Content

Type	Title / Slug	URL	In sitemap
Loading...

Settings

Budgets, routing, password

Budget Limits

Monthly Total ($)

AI Routing Mode

Crawl Delay (ms)

Max Retries

Change Admin Password

New Password

Confirm

Free API Keys Reference

Provider	URL	Free Tier	.env Key
Google Gemini ★	aistudio.google.com	~1M tokens/day	GEMINI_API_KEY
Groq ★	console.groq.com	14,400 req/day free	GROQ_API_KEY
Cohere	dashboard.cohere.com	1,000 req/month	COHERE_API_KEY
Semantic Scholar ★	semanticscholar.org/product/api	FREE, key = 10x faster	SEMANTIC_SCHOLAR_API_KEY

Professors DB

Loading...

Extract Info Enter name, website URL, Google Scholar, or ORCID link — auto-fills data

Search

Country

Verification

Show on site

	Name	University	Country	H-idx	Cit.	Scholar	Lab	Site	Actions
Loading...

Jobs DB

Loading...

	Title	Institution	Country	Type	Deadline	Fields	Apply	Actions
Loading...

—

Conferences DB

Loading...

Series

Loading...

Select a conference series from the list

Labs DB

Loading...

Extract Lab Info Enter a lab URL or name — auto-extracts description, PI, research fields

	Lab Name	University	Director / PI	Country	Fields	Open	Lab	Actions
Loading...

—

Universities DB

Loading...

Rank	University	Country	City	Fields	Profs	Website
Loading...

—

Faculty Agent

Enrichment, verification, and pipeline for professor data

Pipeline Sequence

Cycle -

Loading pipeline state...

Data Quality Metrics

Loading...

Visibility

-

Enricher v2 - Dry Test

OpenAlex, ORCID, Semantic Scholar, 0.85 name threshold, quality scoring

Idle

Country

Count

Agent Quick Controls

rt-enricher, rt-verifier, rt-pipeline

Live PM2 Processes

Loading agents...

Enricher Progress

-

--

Verifier Progress

-

ResearchGate URLs always show as dead - normal.

Watchdog Status

-

Agent Logs

Select an agent above to view logs...

Conference Agent

Enrichment, verification, and pipeline for conf data

Conference Enrichment

Update CFP deadlines, dates, venue info using Gemini AI

Idle

AI Provider

-

URL Verification

HTTP-check all conference website URLs for liveness

Idle

-

Pipeline Steps

1

Fetch CFP Data

Pull new CFP submissions from monitored sources

Ready

2

Enrich with Gemini

Extract deadline, dates, venue using AI

Ready

3

Verify URLs

HTTP-check all conference website URLs

Ready

4

Score and Rank

Score relevance for HCI/VR/Haptics domain

Ready

5

Deploy to Live

Rebuild data_conf.js and push to frontend

Ready

Agent Logs

Select an agent above to view logs...

Jobs Agent

Enrichment, verification, and pipeline for jobs data

Jobs Enrichment

Extract fields, supervisor, lab details from job descriptions using AI

Idle

AI Provider

-

Apply URL Verification

HTTP-check all job application URLs to detect expired listings

Idle

-

Pipeline Steps

1

Crawl Job Sources

Scrape jobs.ac.uk, EURAXESS, Nature, FindAPhD

Ready

2

Enrich Job Details

Extract fields, supervisor, lab from description using AI

Ready

3

Verify Apply URLs

HTTP-check all application URLs for liveness

Ready

4

Mark Expired

Flag jobs past deadline as inactive

Ready

5

Deploy to Live

Rebuild data_job.js and push to frontend

Ready

Agent Logs

Select an agent above to view logs...

Labs Agent

Enrichment, verification, and pipeline for labs data

Labs Enrichment

Enrich lab profiles with director info, research focus, and members using AI

Idle

AI Provider

-

Lab URL Verification

HTTP-check all lab website URLs and open positions pages

Idle

-

Pipeline Steps

1

Discover Labs

Identify new HCI/VR/Haptics labs from publications

Ready

2

Enrich Profiles

Extract director, focus, members using AI

Ready

3

Verify Lab URLs

HTTP-check all lab website URLs

Ready

4

Update Open Positions

Check lab sites for open PhD/postdoc positions

Ready

5

Deploy to Live

Rebuild data_lab.js and push to frontend

Ready

Agent Logs

Select an agent above to view logs...

University Agent

Enrichment, verification, and pipeline for uni data

Department Enrichment

Discover and enrich HCI/VR/Haptics departments at tracked universities

Idle

AI Provider

-

Department Page Verification

HTTP-check all university department page URLs for liveness

Idle

-

Pipeline Steps

1

Discover Departments

Find CS/ECE departments at tracked universities

Ready

2

Enrich Department Data

Extract HCI/VR faculty count, research focus using AI

Ready

3

Verify Dept Pages

HTTP-check department website URLs

Ready

4

Link to Faculty

Associate department entries with professor records

Ready

5

Deploy to Live

Update university metadata in frontend build

Ready

Agent Logs

Select an agent above to view logs...

What is ResearchTheta?

ResearchTheta is a research intelligence platform for HCI, VR, and Haptics academics. It helps PhD applicants and researchers find supervisors, open jobs, conferences, and labs in one place.

The live website at researchtheta.com is a single HTML file (~2MB) with all data baked in -- no database queries on page load.

This admin panel at admin.researchtheta.com manages everything: crawl new data, enrich it with AI agents, score quality, deploy to live site.

Quick Reference

Live site	`researchtheta.com`
Admin panel	`admin.researchtheta.com`
VPS IP	`187.77.159.108`
Admin port	`4000` (PM2: rr-admin)
Hosting	Hostinger (HTML file)
Auth	Firebase (researchradar-322)
SSH	`ssh root@187.77.159.108`

Data Snapshot

Professors	2,100+
Conferences	225+
Labs	111+
Jobs	40+
Countries	29

PM2 Processes Running on VPS

Process	What it does	Status
`rr-admin`	This admin panel (port 4000, Nginx proxy)	Always on
`rt-enricher`	Adds h-index, citations, ORCID to professors via Semantic Scholar, OpenAlex, DBLP	24/7
`rt-verifier`	HTTP-checks all professor lab & scholar URLs for liveness	24/7
`rt-pipeline`	Full orchestration cycle: Enrich to Verify to Score to Deploy	Scheduled
`rt-watchdog`	Pings live site every 30 min, alerts on downtime	Always on

Complete Function Audit -- Every Button and API Call

WorkingStubMissingNot Impl.

ResearchTheta Admin Panel
+-- Overview
|   +-- Dashboard ....................................... Working
|       +-- loadDashboard() -- GET /api/stats ........... Working
|       +-- loadDashAgents() -- GET /api/agents ......... Working
|       +-- loadDataFreshness() -- GET /api/data/freshness Working
+-- AI and Data
|   +-- AI Providers .................................... Working
|   |   +-- loadProviders() -- GET /api/providers ....... Working
|   |   +-- saveProvider() -- PUT /api/providers/:id .... Working
|   |   +-- testProvider() -- POST /api/providers/:id/test Working
|   +-- Task Routing .................................... Working
|   |   +-- loadTaskRouting() -- GET /api/task-routing .. Working
|   |   +-- saveTaskRouting() -- PUT /api/task-routing .. Working
|   +-- Validation ...................................... Working
|       +-- runValidation() -- POST /api/validate/run ... Working
|       +-- stopValidation() -- POST /api/validate/stop . Working
+-- Crawlers
|   +-- Faculty Crawl ................................... Working
|   |   +-- runProfCrawler() ........................... Working
|   |   +-- stopProfCrawler() .......................... Working
|   |   +-- runEnrichment() -- POST /api/enrich/scholars  Working  [also in Faculty Agent]
|   |   +-- stopEnrichment() -- POST /enrich/scholars/stop Working  [bug fixed: was conf-log]
|   |   +-- runStrategyAction(score) -- POST /api/strategy/score  Fixed  [route added]
|   |   +-- runStrategyAction(deploy) -- POST /api/strategy/deploy Fixed  [route added]
|   +-- Conference Crawl ................................ Working
|   |   +-- runConfCrawler() -- POST /api/crawl/conferences Working
|   |   +-- runCommitteeCrawler() -- POST /api/crawl/committees Working
|   |   +-- stopCommitteeCrawler() ..................... Working
|   +-- Jobs Crawl ...................................... Working
|   |   +-- runCrawler() -- POST /api/crawl/run ......... Working
|   |   +-- scrapeJobUrl() -- POST /api/crawl/scrape-url  Working
|   +-- Labs Crawl ...................................... STUB
|   |   +-- runLabsCrawler() -- (no API call) ........... STUB: coming soon
|   +-- University Crawl ................................ STUB
|   |   +-- runUniCrawler() -- (no API call) ............ STUB: coming soon
|   |   +-- scrapeSingleUrl() -- POST /api/crawl/scrape-url Working
|   +-- Conf URL Check -- POST /api/conf/check .......... Working
|   +-- Schedule -- GET/PUT /api/cron ................... Working
+-- Agents
|   +-- Faculty Agent ................................... Working
|   |   +-- loadAgents() -- GET /api/agents ............. Working
|   |   +-- loadAgentProgress() -- GET /api/agents/progress Working
|   |   +-- loadPipelineStatus() -- GET /api/pipeline/status Working
|   |   +-- loadQualityStats() -- GET /api/strategy/quality Working
|   |   +-- loadWatchdogStatus() -- GET /api/watchdog/status Working
|   |   +-- loadAgentLog() -- GET /api/agents/:name/logs  Fixed  [route added]
|   |   +-- agentAction() -- POST /api/agents/:name/action Working
|   |   +-- runFullPipeline() via agentAction(restart) .. Working
|   |   +-- runStrategyAction(score) -- POST /api/strategy/score Fixed
|   |   +-- runStrategyAction(deploy) -- POST /api/strategy/deploy Fixed
|   |   +-- startDryRun() -- POST /api/enrich/v2/dry-run  Working
|   |   +-- startFullEnrichV2() -- POST /api/enrich/v2/start Working
|   +-- Conference / Jobs / Labs / University Agents .... Partial
|       +-- loadEntityLog() -- GET /api/agents/:name/logs Fixed
|       +-- runEntityEnrich() -- POST /api/enrich/:entity Stub response
|       +-- stopEntityEnrich() ....................... Stub response
|       +-- runEntityVerify() -- POST /api/verify/:entity Stub response
|       +-- stopEntityVerify() ....................... Stub response
|       +-- runEntityPipeline() -- POST /api/pipeline/run Stub response
+-- Databases
|   +-- Professors -- GET /api/professors ............... Working
|   +-- Jobs -- GET /api/data/jobs ...................... Working
|   +-- Conferences -- GET /api/data/conferences ........ Working
|   +-- Labs -- GET /api/data/labs ...................... Working
|   +-- DB Utilities -- GET /api/data/counts ............ Working
+-- Management
    +-- Documentation ................................... Working
    +-- SEO Pages -- GET /api/seo/stats ................. Working
    +-- Deploy -- POST /api/deploy/rebuild+hostinger ..... Working
    +-- Logs -- GET /api/logs ........................... Working
    +-- Settings -- GET/PUT /api/settings ............... Working

Conflicts and Design Issues

1. Scholar Enrichment on Crawl Faculty page

runEnrichment() appears on both Crawl/Faculty AND Agents/Faculty. Rule: Crawl = find new data. Agent = enrich existing data. The Crawl Faculty version is a convenience shortcut -- both call the same API endpoint and work correctly.

2. Quality Score / Deploy on Crawl Faculty page

runStrategyAction() is on Crawl Faculty as "Quick Actions". These are agent/deploy operations. They were broken (missing routes) -- now fixed. Conceptually they belong in Agents and Deploy pages.

3. loadPipelineStatus() defined twice

Two definitions in admin.html (from migration). The second overwrites the first. Both identical -- no visible bug but dead code.

What Was Fixed in This Audit

FixedGET /api/agents/:name/logs -- was missing, Agent Logs now work

FixedPOST /api/strategy/score -- was missing, Quality Scorer now works

FixedPOST /api/strategy/deploy -- was missing, Safe Deploy now works

FixedPOST /api/pipeline/run -- entity pipeline buttons no longer crash

FixedPOST /api/enrich/:entity -- clear message instead of HTML error

FixedPOST /api/verify/:entity -- same

FixedstopEnrichment() -- was writing to conf-log (wrong ID), fixed

FixedOld page-agents DOM duplicate -- removed

What Still Needs Backend Implementation

Feature	Page	Required API	What to build
Labs Crawler	Crawl / Labs	POST /api/crawl/labs	Python crawler script, PM2 process or inline Node exec
University Crawler	Crawl / Universities	POST /api/crawl/universities	Python crawler script scraping university department pages
Conference Enrichment	Agents / Conferences	POST /api/enrich/conf	Script to update CFP deadlines, dates via Gemini AI
Jobs Enrichment	Agents / Jobs	POST /api/enrich/jobs	Script to extract supervisor, field, lab from job description
Labs Enrichment	Agents / Labs	POST /api/enrich/labs	Script to enrich lab profiles with director info, research focus
University Enrichment	Agents / Universities	POST /api/enrich/uni	Script to extract HCI/VR dept info from university pages
Conference Verification	Agents / Conferences	POST /api/verify/conf	HTTP-check all conference URLs
Jobs Verification	Agents / Jobs	POST /api/verify/jobs	HTTP-check all job apply URLs, flag expired
Labs Verification	Agents / Labs	POST /api/verify/labs	HTTP-check all lab website URLs
University Verification	Agents / Universities	POST /api/verify/uni	HTTP-check all department page URLs
Quality Scorer script	Faculty Agent	POST /api/strategy/score	Full quality_score.py script (currently restarts rt-pipeline)

Command	What it does
`pm2 list`	Show all running processes
`pm2 restart rr-admin`	Restart admin panel
`pm2 logs rt-enricher --lines 100`	View enricher log
`pm2 monit`	Live dashboard of all PM2 processes
`systemctl status nginx`	Check Nginx
`psql -U postgres -d researchradar`	Open database shell
`df -h`	Check disk space
`htop`	Live CPU/memory

Source	Data added	Rate
Semantic Scholar	h-index, citations, ss_id	100/5min
OpenAlex	citations fallback	10/sec
ORCID	orcid_url, affiliation	24/sec
DBLP	publications	Unlimited
GPT-4o-mini	URL verification	~$0.50/1k
Gemini Flash	AI enrichment	60/min free
Groq	AI enrichment	30/min free

Method	Endpoint	Status	Description
Professors
GET	/api/professors	Live	List with q, country, page, limit
PUT	/api/professors/:id	Live	Update fields
POST	/api/professors	Live	Add professor
DEL	/api/data/professors/:id	Live	Delete professor
Agents and PM2
GET	/api/agents	Live	PM2 process list + status
GET	/api/agents/progress	Live	Enricher/verifier progress
POST	/api/agents/:name/action	Live	start/stop/restart PM2
GET	/api/agents/:name/logs	Fixed	Last N lines from PM2 log file
Pipeline and Quality
GET	/api/pipeline/status	Live	Pipeline cycle state
GET	/api/strategy/quality	Live	shown/hidden/unscored counts
POST	/api/strategy/score	Fixed	Trigger quality scorer
POST	/api/strategy/deploy	Fixed	Trigger safe deploy
GET	/api/watchdog/status	Live	Watchdog last check
Enrichment and Crawl
POST	/api/enrich/scholars	Live	Scholar enrichment (faculty)
POST	/api/enrich/v2/dry-run	Live	Enricher v2 dry test
POST	/api/enrich/v2/start	Live	Full enricher v2 run
POST	/api/enrich/:entity	Stub	conf/jobs/labs/uni -- shows not_implemented message
POST	/api/verify/:entity	Stub	Entity verification -- shows not_implemented message
POST	/api/pipeline/run	Stub	Entity pipeline -- faculty only implemented
POST	/api/crawl/professors	Live	Professor crawler
POST	/api/crawl/conferences	Live	Conference crawler
POST	/api/crawl/run	Live	Jobs crawler
POST	/api/crawl/committees	Live	Committee crawler
GET	/api/crawl/sources	Live	Crawl sources list

Dashboard

AI Providers

Task Routing

Professor Validation

Faculty Control

Conference Control

Jobs Control

Labs Control

University Control

Conference URL Check

Schedule

Data Manager

SEO Pages

Deploy Control

Logs Viewer

Backups

SEO & Publish

Settings

Professors DB

Jobs DB

Conferences DB

—

Labs DB

Universities DB

Faculty Agent

Conference Agent

Jobs Agent

Labs Agent

University Agent

Documentation

Edit Professor

Edit

Confirm