DEV: Expose AI spam scanning metrics (#1077)

This should give us a better idea on how our scanner is faring across sites.

```
# HELP discourse_discourse_ai_spam_detection AI spam scanning statistics
# TYPE discourse_discourse_ai_spam_detection counter
discourse_discourse_ai_spam_detection{db="default",type="scanned"} 16
discourse_discourse_ai_spam_detection{db="default",type="is_spam"} 7
discourse_discourse_ai_spam_detection{db="default",type="false_positive"} 1
discourse_discourse_ai_spam_detection{db="default",type="false_negative"} 2
```
This commit is contained in:
Natalie Tay 2025-01-27 11:57:01 +08:00 committed by GitHub
parent ad7bb9bd31
commit fe44d78156
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
4 changed files with 42 additions and 0 deletions

View File

@ -2,3 +2,6 @@ inherit_gem:
rubocop-discourse: stree-compat.yml
RSpec/NamedSubject:
Enabled: false
Style/GlobalVars:
AllowedVariables: [$prometheus_client]

7
about.json Normal file
View File

@ -0,0 +1,7 @@
{
"tests": {
"requiredPlugins": [
"https://github.com/discourse/discourse-prometheus"
]
}
}

View File

@ -0,0 +1,29 @@
# frozen_string_literal: true
module DiscourseAi
module AiModeration
class SpamMetric
def self.update(new_status, reviewable)
ai_spam_log = AiSpamLog.find_by(reviewable:)
return if ai_spam_log.nil?
increment("scanned")
increment("is_spam") if new_status == :approved && ai_spam_log.is_spam
increment("false_positive") if new_status == :rejected && ai_spam_log.is_spam
increment("false_negative") if new_status == :rejected && !ai_spam_log.is_spam
end
private
def self.increment(type, value = 1)
metric = ::DiscoursePrometheus::InternalMetric::Custom.new
metric.name = "discourse_ai_spam_detection"
metric.type = "Counter"
metric.description = "AI spam scanning statistics"
metric.labels = { db: RailsMultisite::ConnectionManagement.current_db, type: }
metric.value = value
$prometheus_client.send_json(metric.to_h)
end
end
end
end

View File

@ -84,6 +84,9 @@ after_initialize do
on(:reviewable_transitioned_to) do |new_status, reviewable|
ModelAccuracy.adjust_model_accuracy(new_status, reviewable)
if DiscourseAi::AiModeration::SpamScanner.enabled?
DiscourseAi::AiModeration::SpamMetric.update(new_status, reviewable)
end
end
if Rails.env.test?