Data Collection
How We Gathered Game Data
The most technically interesting part of the site was how we collected real-time game data.
Knights Online’s client communicates with servers using a binary protocol. Rather than
intercepting packets (which would have violated terms of service), we used a combination of
approaches: in-game surveying, community crowd-sourcing, and parsing publicly available
game files.
The Survey / Scanner System
The core data pipeline relied on a survey system. Player volunteers would
run a lightweight PHP-based “scanner” that accepted manually entered game data
(monster locations, character stats, item data). This data was submitted to our database
through a simple HTTP API and aggregated across all contributors.
The common/scanner.php file handled incoming survey data, validated it, and
inserted or updated the appropriate database records. The monsterhelper and
monsterloc tables stored crowd-sourced monster location data tied to zone IDs.
function process_mob_location($data) {
$zone_id = intval($data['zone']);
$mob_id = intval($data['mob']);
$coord_x = intval($data['x']);
$coord_z = intval($data['z']);
$existing = mysql_query(
"SELECT id, count FROM monsterloc
WHERE zone_id = $zone_id
AND mob_id = $mob_id
AND ABS(x - $coord_x) < 50
AND ABS(z - $coord_z) < 50"
);
if (mysql_num_rows($existing) > 0) {
$row = mysql_fetch_assoc($existing);
mysql_query("UPDATE monsterloc SET count = count + 1 WHERE id = {$row['id']}");
} else {
mysql_query(
"INSERT INTO monsterloc (zone_id, mob_id, x, z, count)
VALUES ($zone_id, $mob_id, $coord_x, $coord_z, 1)"
);
}
}
function normalize_server_name($raw) {
$map = [
'ares' => 'Ares',
'xigenon' => 'Xigenon',
'beramus' => 'Beramus',
'cypher' => 'Cypher',
];
return $map[strtolower($raw)] ?? ucfirst($raw);
}
Item Data Pipeline
Item data presented a unique challenge: Knights Online has thousands of items with complex
stat formulas, enchantment tiers, and class requirements. We extracted this data by parsing
the game client’s data files, cross-referencing with community wikis, and manual testing.
Item rarity was calculated from the enchantment level (+1 through +9) and
item tier. The scanner.php file contained the rarity calculation logic that
colored items in the UI — similar to how World of Warcraft colors gear by quality tier.
function get_item_rarity($enchant, $grade) {
if ($grade >= 9 && $enchant >= 8) {
return 'item-legendary';
} elseif ($grade >= 7 || $enchant >= 7) {
return 'item-epic';
} elseif ($grade >= 5 || $enchant >= 5) {
return 'item-rare';
} elseif ($grade >= 3) {
return 'item-uncommon';
}
return 'item-common';
}
function get_weapon_type($item_id) {
if ($item_id >= 100 && $item_id <= 199) return 'Sword';
if ($item_id >= 200 && $item_id <= 299) return 'Axe';
if ($item_id >= 300 && $item_id <= 399) return 'Staff';
if ($item_id >= 400 && $item_id <= 499) return 'Bow';
return 'Unknown';
}