Bug: actor avatar changes do not notify ActivityPub followers #38

New issue

Closed

opened 2026-06-28 10:56:24 -05:00 by erik · 8 comments

erik commented 2026-06-28 10:56:24 -05:00

Owner

Copy link

Problem

Changing the actor avatar image at https://slugkit.com/settings/actor updates local actor/profile settings but does not notify ActivityPub followers, so remote followers such as Mastodon do not receive/update the new avatar.

Observed remote target:

• mastodon.online/@evcraddock / Mastodon follow relationship for the slugkit.com actor

Expected behavior

When the actor avatar is changed in settings:

• The local actor profile updates.
• An ActivityPub Update for the actor is delivered to followers.
• Remote servers refresh the actor/profile avatar after receiving the update.

Scope

• Inspect actor settings update flow.
• Determine whether actor profile updates currently enqueue/deliver ActivityPub actor Update activities.
• Add delivery for actor profile/avatar changes to followers.
• Ensure the delivered Actor object includes the updated avatar/icon image.
• Use unique Update activity IDs so repeated actor profile changes are not deduped by remote servers.
• Add regression tests for avatar/profile setting updates triggering ActivityPub delivery.

Acceptance criteria

• Updating the avatar through /settings/actor triggers an ActivityPub actor Update delivery to followers.
• The delivered actor object includes the updated avatar/icon URL.
• Repeated actor updates produce unique ActivityPub Update activity IDs.
• Existing actor settings behavior remains unchanged locally.
• Tests cover actor avatar update delivery and serialized payload shape.
• Verify against Mastodon after deploy that the remote avatar refreshes or at least receives a valid actor Update.

## Problem Changing the actor avatar image at `https://slugkit.com/settings/actor` updates local actor/profile settings but does not notify ActivityPub followers, so remote followers such as Mastodon do not receive/update the new avatar. Observed remote target: - `mastodon.online/@evcraddock` / Mastodon follow relationship for the `slugkit.com` actor ## Expected behavior When the actor avatar is changed in settings: - The local actor profile updates. - An ActivityPub `Update` for the actor is delivered to followers. - Remote servers refresh the actor/profile avatar after receiving the update. ## Scope - Inspect actor settings update flow. - Determine whether actor profile updates currently enqueue/deliver ActivityPub actor `Update` activities. - Add delivery for actor profile/avatar changes to followers. - Ensure the delivered Actor object includes the updated avatar/icon image. - Use unique Update activity IDs so repeated actor profile changes are not deduped by remote servers. - Add regression tests for avatar/profile setting updates triggering ActivityPub delivery. ## Acceptance criteria - Updating the avatar through `/settings/actor` triggers an ActivityPub actor `Update` delivery to followers. - The delivered actor object includes the updated avatar/icon URL. - Repeated actor updates produce unique ActivityPub Update activity IDs. - Existing actor settings behavior remains unchanged locally. - Tests cover actor avatar update delivery and serialized payload shape. - Verify against Mastodon after deploy that the remote avatar refreshes or at least receives a valid actor Update.

erik added the

settings

actor

activitypub

bug

mastodon

status:active

priority:medium

labels 2026-06-28 10:56:24 -05:00

erik commented 2026-06-28 11:02:02 -05:00

Author

Owner

Copy link

Synced from todu comment by @todu on 2026-06-28T16:01:16.444Z

Additional requirement: actor settings logging

While investigating via k3s pod logs, we found successful actor/settings changes are not currently logged. That makes it hard to verify whether an avatar change happened, whether ActivityPub delivery was attempted, or whether a remote follower update failed.

Add logging as part of this task:

• Log successful actor settings/profile saves, including which fields changed (e.g. display name, summary, avatar/icon URL) without logging sensitive data.
• Log whether an ActivityPub actor Update delivery is enqueued/attempted after actor settings changes.
• Log delivery success/failure with enough context to debug follower notification issues, including actor username and recipient/follower count; avoid logging secrets/private keys.
• Ensure logs are structured consistently with existing API/request logs.

Acceptance criteria addition:

• Changing the actor avatar through /settings/actor produces a structured log entry for the settings change.
• The same operation produces structured ActivityPub delivery logs for the actor update attempt/result.

_Synced from todu comment by @todu on 2026-06-28T16:01:16.444Z_ ### Additional requirement: actor settings logging While investigating via k3s pod logs, we found successful actor/settings changes are not currently logged. That makes it hard to verify whether an avatar change happened, whether ActivityPub delivery was attempted, or whether a remote follower update failed. Add logging as part of this task: - Log successful actor settings/profile saves, including which fields changed (e.g. display name, summary, avatar/icon URL) without logging sensitive data. - Log whether an ActivityPub actor `Update` delivery is enqueued/attempted after actor settings changes. - Log delivery success/failure with enough context to debug follower notification issues, including actor username and recipient/follower count; avoid logging secrets/private keys. - Ensure logs are structured consistently with existing API/request logs. Acceptance criteria addition: - Changing the actor avatar through `/settings/actor` produces a structured log entry for the settings change. - The same operation produces structured ActivityPub delivery logs for the actor update attempt/result.

erik added

status:inprogress

and removed

status:active

labels 2026-06-28 11:07:39 -05:00

erik commented 2026-06-28 11:13:19 -05:00

Author

Owner

Copy link

Synced from todu comment by @todu on 2026-06-28T16:13:04.328Z

Implementation PR #39

Opened and reviewed approved:

erik/slugkit.com#39 — Notify ActivityPub followers on actor updates

Changes:

• /settings/actor saves now call ActivityPub actor Update delivery after saving local actor/account settings.
• Actor update payload includes the serialized actor profile with avatar icon, banner image, profile attachments, inbox/outbox/followers/following/featured URLs, and shared inbox endpoint.
• Actor update activity IDs include the actor updatedAt timestamp so repeated profile/avatar changes are not deduped by remotes.
• Added structured logs for:
  • successful actor settings saves and changed field names,
  • actor update delivery started,
  • actor update delivery queued,
  • skipped delivery when ActivityPub is disabled or there are no followers.
• Settings routes now receive the ActivityPub queue so tests and production use the same delivery path.
• Regression test covers actor update enqueueing, avatar URL serialization, follower inbox targeting, logging, and repeated unique update IDs.

Verification passed:

• npm run typecheck
• npm test
• npm run build

Process note:

• I initially committed directly to main by mistake. I repaired this without force-pushing by reverting that commit on main, then cherry-picking the implementation onto task-d213c262-actor-update-delivery and opening PR #39 normally. Current PR contains the intended changes.

_Synced from todu comment by @todu on 2026-06-28T16:13:04.328Z_ ### Implementation PR #39 Opened and reviewed approved: `erik/slugkit.com#39` — Notify ActivityPub followers on actor updates Changes: - `/settings/actor` saves now call ActivityPub actor `Update` delivery after saving local actor/account settings. - Actor update payload includes the serialized actor profile with avatar `icon`, banner image, profile attachments, inbox/outbox/followers/following/featured URLs, and shared inbox endpoint. - Actor update activity IDs include the actor `updatedAt` timestamp so repeated profile/avatar changes are not deduped by remotes. - Added structured logs for: - successful actor settings saves and changed field names, - actor update delivery started, - actor update delivery queued, - skipped delivery when ActivityPub is disabled or there are no followers. - Settings routes now receive the ActivityPub queue so tests and production use the same delivery path. - Regression test covers actor update enqueueing, avatar URL serialization, follower inbox targeting, logging, and repeated unique update IDs. Verification passed: - `npm run typecheck` - `npm test` - `npm run build` Process note: - I initially committed directly to `main` by mistake. I repaired this without force-pushing by reverting that commit on `main`, then cherry-picking the implementation onto `task-d213c262-actor-update-delivery` and opening PR #39 normally. Current PR contains the intended changes.

erik commented 2026-06-28 11:36:00 -05:00

Author

Owner

Copy link

Synced from todu comment by @todu on 2026-06-28T16:33:34.776Z

Production deploy findings after web-v0.1.24

The new logging confirmed the settings path and delivery path were running, but Mastodon rejected delivery.

Observed logs:

• slugkit settings: actor saved { actorUsername: 'slug', actorId: 1, changedFields: [ 'avatar' ] }
• slugkit ActivityPub actor update delivery started ... followerCount: 1
• slugkit ActivityPub actor update delivery queued ... followerCount: 1
• ActivityPub outbound delivery failed ... 401 Unauthorized ... {"error":"Public key not found for key https://slugkit.com/users/slug#multikey-1"}

Additional deployment note:

• The brief site downtime was from the deployment using Recreate, plus Flux/image automation reconciliation causing extra pod churn. There was a zero-ready-pod window while the new pod started.

Hotfix PR opened and reviewed approved:

erik/slugkit.com#40 — Use ActivityPub main key for HTTP signatures

Change:

• Sign outbound ActivityPub HTTP delivery with #main-key instead of #multikey-1.
• Mastodon already sees publicKey.id = https://slugkit.com/users/slug#main-key in the actor document.
• Tests updated to assert delivery signatures use #main-key and not #multikey-1.

Verification passed:

• npm run typecheck
• npm test -- --run src/federation/__tests__/delivery.test.ts src/federation/__tests__/routes.test.ts
• npm test
• npm run build

Next after merge/release/deploy:

• Change actor avatar again.
• Verify pod logs do not show Mastodon 401 for #multikey-1.
• Verify Mastodon accepts/applies the actor Update.

_Synced from todu comment by @todu on 2026-06-28T16:33:34.776Z_ ### Production deploy findings after `web-v0.1.24` The new logging confirmed the settings path and delivery path were running, but Mastodon rejected delivery. Observed logs: - `slugkit settings: actor saved { actorUsername: 'slug', actorId: 1, changedFields: [ 'avatar' ] }` - `slugkit ActivityPub actor update delivery started ... followerCount: 1` - `slugkit ActivityPub actor update delivery queued ... followerCount: 1` - `ActivityPub outbound delivery failed ... 401 Unauthorized ... {"error":"Public key not found for key https://slugkit.com/users/slug#multikey-1"}` Additional deployment note: - The brief site downtime was from the deployment using `Recreate`, plus Flux/image automation reconciliation causing extra pod churn. There was a zero-ready-pod window while the new pod started. Hotfix PR opened and reviewed approved: `erik/slugkit.com#40` — Use ActivityPub main key for HTTP signatures Change: - Sign outbound ActivityPub HTTP delivery with `#main-key` instead of `#multikey-1`. - Mastodon already sees `publicKey.id = https://slugkit.com/users/slug#main-key` in the actor document. - Tests updated to assert delivery signatures use `#main-key` and not `#multikey-1`. Verification passed: - `npm run typecheck` - `npm test -- --run src/federation/__tests__/delivery.test.ts src/federation/__tests__/routes.test.ts` - `npm test` - `npm run build` Next after merge/release/deploy: - Change actor avatar again. - Verify pod logs do not show Mastodon 401 for `#multikey-1`. - Verify Mastodon accepts/applies the actor Update.

erik commented 2026-06-28 11:52:48 -05:00

Author

Owner

Copy link

Synced from todu comment by @todu on 2026-06-28T16:48:33.936Z

Correction: PR #40 was wrong

Closed task task-becaf72b already established the key behavior:

• Mastodon 4.7 stores the Fedify 2 RSA key as https://slugkit.com/users/slug#multikey-1.
• Outbound delivery signed with #main-key was rejected by Mastodon.
• The verified fix was to sign outbound ActivityPub delivery with #multikey-1.

PR #40 changed signing back to #main-key, which regressed that known fix. Production logs after web-v0.1.25 confirmed Mastodon rejected #main-key too.

Opened and reviewed approved:

erik/slugkit.com#41 — Revert main-key ActivityPub signature regression

Verification passed:

• npm run typecheck
• npm test -- --run src/federation/__tests__/delivery.test.ts src/federation/__tests__/routes.test.ts
• npm test
• npm run build

Next: merge/release/deploy #41 to restore known-working outbound signing, then investigate actor Update-specific failure with #multikey-1 without changing the global signing key behavior again.

_Synced from todu comment by @todu on 2026-06-28T16:48:33.936Z_ ### Correction: PR #40 was wrong Closed task `task-becaf72b` already established the key behavior: - Mastodon 4.7 stores the Fedify 2 RSA key as `https://slugkit.com/users/slug#multikey-1`. - Outbound delivery signed with `#main-key` was rejected by Mastodon. - The verified fix was to sign outbound ActivityPub delivery with `#multikey-1`. PR #40 changed signing back to `#main-key`, which regressed that known fix. Production logs after `web-v0.1.25` confirmed Mastodon rejected `#main-key` too. Opened and reviewed approved: `erik/slugkit.com#41` — Revert main-key ActivityPub signature regression Verification passed: - `npm run typecheck` - `npm test -- --run src/federation/__tests__/delivery.test.ts src/federation/__tests__/routes.test.ts` - `npm test` - `npm run build` Next: merge/release/deploy #41 to restore known-working outbound signing, then investigate actor Update-specific failure with `#multikey-1` without changing the global signing key behavior again.

erik commented 2026-06-28 11:58:22 -05:00

Author

Owner

Copy link

Synced from todu comment by @todu on 2026-06-28T16:58:11.966Z

Revert released/deployed

Merged PR #41, released web-v0.1.26, and deployed evcraddock/slugkit-com:0.1.26.

Details:

• Commit: cb11340
• Forgejo Actions: #32 succeeded
• k3s rollout: succeeded
• Current deployment image: evcraddock/slugkit-com:0.1.26

This restores the known-working outbound ActivityPub signing key behavior from task-becaf72b: delivery signatures use #multikey-1, not #main-key.

Task remains open because actor avatar Update delivery still needs investigation/fix without regressing the global signing key behavior.

_Synced from todu comment by @todu on 2026-06-28T16:58:11.966Z_ ### Revert released/deployed Merged PR #41, released `web-v0.1.26`, and deployed `evcraddock/slugkit-com:0.1.26`. Details: - Commit: `cb11340` - Forgejo Actions: `#32` succeeded - k3s rollout: succeeded - Current deployment image: `evcraddock/slugkit-com:0.1.26` This restores the known-working outbound ActivityPub signing key behavior from `task-becaf72b`: delivery signatures use `#multikey-1`, not `#main-key`. Task remains open because actor avatar Update delivery still needs investigation/fix without regressing the global signing key behavior.

erik commented 2026-06-28 12:15:28 -05:00

Author

Owner

Copy link

Synced from todu comment by @todu on 2026-06-28T17:14:34.189Z

Replacement fix PR #42

Opened and reviewed approved:

erik/slugkit.com#42 — Send actor updates with keyed Fedify actor object

What changed from the broken PR #39 implementation:

• Removed the separate partial actor object construction path from actor Update delivery.
• Extracted the existing Fedify actor dispatcher serialization into buildLocalActivityPubActorObject(...).
• The actor dispatcher and actor Update delivery now use the same helper.
• Actor Update payload now includes both key forms that the actor document exposes:
  • publicKey / #main-key / publicKeyPem
  • assertionMethod / #multikey-1 / publicKeyMultibase
• Outbound HTTP signatures remain on the known-working #multikey-1 path from task-becaf72b.
• Actor save/delivery logging is preserved.

Verification passed:

• npm run typecheck
• npm test -- --run src/routes/__tests__/admin-activitypub.test.ts src/federation/__tests__/routes.test.ts src/federation/__tests__/delivery.test.ts
• npm test
• npm run build

Important deploy note:

• Production Mastodon may have cached bad actor/key state from the broken update attempts. After deploying PR #42, first publish a test note to check normal fanout. If Mastodon still returns Public key not found, force a remote actor refetch/re-follow flow before assuming the new payload is still wrong.

_Synced from todu comment by @todu on 2026-06-28T17:14:34.189Z_ ### Replacement fix PR #42 Opened and reviewed approved: `erik/slugkit.com#42` — Send actor updates with keyed Fedify actor object What changed from the broken PR #39 implementation: - Removed the separate partial actor object construction path from actor Update delivery. - Extracted the existing Fedify actor dispatcher serialization into `buildLocalActivityPubActorObject(...)`. - The actor dispatcher and actor Update delivery now use the same helper. - Actor Update payload now includes both key forms that the actor document exposes: - `publicKey` / `#main-key` / `publicKeyPem` - `assertionMethod` / `#multikey-1` / `publicKeyMultibase` - Outbound HTTP signatures remain on the known-working `#multikey-1` path from `task-becaf72b`. - Actor save/delivery logging is preserved. Verification passed: - `npm run typecheck` - `npm test -- --run src/routes/__tests__/admin-activitypub.test.ts src/federation/__tests__/routes.test.ts src/federation/__tests__/delivery.test.ts` - `npm test` - `npm run build` Important deploy note: - Production Mastodon may have cached bad actor/key state from the broken update attempts. After deploying PR #42, first publish a test note to check normal fanout. If Mastodon still returns `Public key not found`, force a remote actor refetch/re-follow flow before assuming the new payload is still wrong.

erik commented 2026-06-28 12:43:33 -05:00

Author

Owner

Copy link

Synced from todu comment by @todu on 2026-06-28T17:38:37.371Z

PR #42 released/deployed

Merged PR #42, released web-v0.1.27, and deployed evcraddock/slugkit-com:0.1.27.

Details:

• Commit: 9db5e5b
• Forgejo Actions: #33 succeeded
• k3s rollout: succeeded
• Current deployment image: evcraddock/slugkit-com:0.1.27

Next verification order:

1. Publish a new test note with slug and verify normal Mastodon fanout.
2. If normal fanout still fails with Public key not found, treat Mastodon remote actor/key cache as poisoned and force refetch/re-follow.
3. Only after normal fanout works, test actor avatar Update delivery again.

_Synced from todu comment by @todu on 2026-06-28T17:38:37.371Z_ ### PR #42 released/deployed Merged PR #42, released `web-v0.1.27`, and deployed `evcraddock/slugkit-com:0.1.27`. Details: - Commit: `9db5e5b` - Forgejo Actions: `#33` succeeded - k3s rollout: succeeded - Current deployment image: `evcraddock/slugkit-com:0.1.27` Next verification order: 1. Publish a new test note with `slug` and verify normal Mastodon fanout. 2. If normal fanout still fails with `Public key not found`, treat Mastodon remote actor/key cache as poisoned and force refetch/re-follow. 3. Only after normal fanout works, test actor avatar Update delivery again.

erik 2026-06-28 13:56:14 -05:00

• closed this issue
• added
  status:done
  and removed
  status:inprogress
  labels

erik commented 2026-06-28 13:56:14 -05:00

Author

Owner

Copy link

Synced from todu comment by @todu on 2026-06-28T18:55:20.462Z

Completed and verified

Released/deployed fixes through web-v0.1.32.

What changed:

• /settings/actor saves trigger ActivityPub actor Update delivery to followers with structured logs.
• Actor Update payload uses the same local actor serialization path as the actor document and includes avatar/icon URL.
• Update activity IDs include actor.updatedAt, so repeated actor saves produce unique IDs.
• Mastodon 4.7 key-cache issue fixed by switching the public actor key shape to legacy publicKey only, served at /activitypub/keys/{actor}/main-key, and signing delivery with acct:{actor}@{domain} so Mastodon refreshes the cached account key correctly.

Verification:

• PRs merged/released/deployed: #39, #42, #43, #44, #45, #46, #47.
• Final deployed image: evcraddock/slugkit-com:0.1.32.
• Live actor document: publicKey.id = https://slugkit.com/activitypub/keys/slug/main-key; assertionMethod omitted.
• Mastodon refollow verified: relationship is following: true, requested: false.
• Published test note activitypub-test-20260628185052; Mastodon account statuses show it as latest status.
• Saved /settings/actor; logs show:
  • slugkit settings: actor saved
  • slugkit ActivityPub actor update delivery started
  • slugkit ActivityPub actor update delivery queued
  • no ActivityPub delivery failure/401 afterward.

Tests run on final changes:

• npm run typecheck
• focused ActivityPub/settings tests
• npm test
• npm run build

_Synced from todu comment by @todu on 2026-06-28T18:55:20.462Z_ ### Completed and verified Released/deployed fixes through `web-v0.1.32`. What changed: - `/settings/actor` saves trigger ActivityPub actor `Update` delivery to followers with structured logs. - Actor Update payload uses the same local actor serialization path as the actor document and includes avatar/icon URL. - Update activity IDs include `actor.updatedAt`, so repeated actor saves produce unique IDs. - Mastodon 4.7 key-cache issue fixed by switching the public actor key shape to legacy `publicKey` only, served at `/activitypub/keys/{actor}/main-key`, and signing delivery with `acct:{actor}@{domain}` so Mastodon refreshes the cached account key correctly. Verification: - PRs merged/released/deployed: #39, #42, #43, #44, #45, #46, #47. - Final deployed image: `evcraddock/slugkit-com:0.1.32`. - Live actor document: `publicKey.id = https://slugkit.com/activitypub/keys/slug/main-key`; `assertionMethod` omitted. - Mastodon refollow verified: relationship is `following: true`, `requested: false`. - Published test note `activitypub-test-20260628185052`; Mastodon account statuses show it as latest status. - Saved `/settings/actor`; logs show: - `slugkit settings: actor saved` - `slugkit ActivityPub actor update delivery started` - `slugkit ActivityPub actor update delivery queued` - no ActivityPub delivery failure/401 afterward. Tests run on final changes: - `npm run typecheck` - focused ActivityPub/settings tests - `npm test` - `npm run build`

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

main

web-v0.1.34

web-v0.1.33

web-v0.1.32

web-v0.1.31

web-v0.1.30

web-v0.1.29

web-v0.1.28

web-v0.1.27

web-v0.1.26

web-v0.1.25

web-v0.1.24

web-v0.1.23

web-v0.1.22

web-v0.1.21

web-v0.1.20

web-v0.1.19

web-v0.1.18

web-v0.1.17

web-v0.1.16

web-v0.1.15

web-v0.1.14

web-v0.1.13

web-v0.1.12

web-v0.1.11

web-v0.1.10

web-v0.1.9

web-v0.1.8

web-v0.1.7

web-v0.1.6

web-v0.1.5

web-v0.1.4

web-v0.1.3

web-v0.1.2

web-v0.1.2-rc.2

web-v0.1.2-rc.1

web-v0.1.1

web-v0.1.0

Labels

Clear labels   

activitypub

  

actor

  

admin

  

ai-safety

  

article

  

articles

  

atproto

  

bug

  

cli

  

configuration

  

content

  

dev-environment

  

docker

  

enhancement

  

feature

  

federation

  

fediverse

  

forgejo

  

garage

  

health

  

how-to

  

logging

  

markdown

  

mastodon

  

media

  

navigation

  

opengraph

  

passkeys

  

preview

  

priority:medium

  

production

  

release

  

rendering

  

s3

  

settings

  

setup

  

status:active

  

status:done

  

status:inprogress

  

technical

  

webauthn

No labels

activitypub

actor

admin

ai-safety

article

articles

atproto

bug

cli

configuration

content

dev-environment

docker

enhancement

feature

federation

fediverse

forgejo

garage

health

how-to

logging

markdown

mastodon

media

navigation

opengraph

passkeys

preview

priority:medium

production

release

rendering

s3

settings

setup

status:active

status:done

status:inprogress

technical

webauthn

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

erik/slugkit.com#38

No description provided.