Automated Scoring for Bittensor Validators

Python library for evaluating validator responses using multiple scoring metrics including BLEU, ROUGE, and Shapley values with real blockchain data integration and MCP server for AI assistant integration.

Python 3.8+ MCP Server MIT License Production Ready
Get Started MCP Server View on GitHub
valiscore mcp-server 
$ valiscore mcp-server --network finney

MCP Server Started:
================================================================================
Server: ValiScore MCP Server
Network: finney
Available Tools: 8
Available Resources: 3
Status: Ready for AI assistant integration

Tools Available:
• score_responses - Score multiple responses
• analyze_subnet - Real subnet analysis
• compare_subnets - Multi-subnet comparison
• monitor_subnet - Real-time monitoring
• calculate_shapley - Fair contribution analysis

Analysis Tools

Real-time blockchain data analysis with industry-standard metrics and advanced fairness analysis

MCP Server Integration

Model Context Protocol server for seamless AI assistant integration with real-time subnet analysis and scoring tools.

Real Blockchain Data

Connect directly to Bittensor networks (finney, test, local) for live validator analysis and performance tracking.

BLEU Scoring

Industry-standard BLEU score for text similarity evaluation with smoothing functions for edge cases.

ROUGE Metrics

ROUGE-1, ROUGE-2, and ROUGE-L scoring for detailed text quality assessment.

Shapley Values

Fair contribution assessment for multiple miners using game theory principles.

CLI Interface

Command-line interface for subnet analysis, monitoring, and batch processing workflows.

MCP Server for AI Assistants

Integrate ValiScore directly into your AI assistant workflows with the Model Context Protocol

8 Available Tools

Score responses, analyze subnets, monitor performance, and calculate Shapley values

3 Resource Types

Access subnet information, validator data, and analysis results as queryable resources

Multi-Network Support

Connect to finney, test, or local Bittensor networks with real-time data

Configurable

Customize network settings, authentication, and tool parameters

Quick Start

# Start MCP server
valiscore mcp-server --network finney

# Show available configuration
valiscore mcp-config-show

# Connect from your AI assistant
# Tools: score_responses, analyze_subnet, compare_subnets
# Resources: subnet_info, validator_data, analysis_results

Quick Installation

Get started with ValiScore in minutes

1

Clone Repository

git clone https://github.com/sonoran-softworks/valiscore.git
cd valiscore
2

Install Dependencies

pip install -e .
3

Run Demo

valiscore demo

Simple Usage

Powerful analysis with minimal setup

Subnet Analysis

# Analyze a real Bittensor subnet
valiscore analyze-subnet --subnet-id 1

# Compare multiple subnets
valiscore compare-subnets --subnet-ids 1,2,3

# Monitor subnet in real-time
valiscore monitor-subnet --subnet-id 1 --interval 60

Python API

from valiscore import ValidatorAnalyzer

analyzer = ValidatorAnalyzer(network="finney")
analysis = await analyzer.analyze_subnet(netuid=1)

print(f"Validators: {analysis['validator_count']}")
print(f"Average Performance: {analysis['average_performance']:.3f}")

MCP Server

# Start MCP server for AI assistant integration
valiscore mcp-server --network finney

# Show MCP configuration
valiscore mcp-config-show

# Available tools: score_responses, analyze_subnet, 
# compare_subnets, monitor_subnet, calculate_shapley

Documentation & Resources

Everything you need to get started with ValiScore

API Reference

Complete API documentation with examples and usage patterns.

View API Docs

MCP Documentation

Learn how to integrate ValiScore with AI assistants using the MCP protocol.

MCP Guide