Migrating back from Firefish to Misskey

Firefish isn't really sparking joy, so I'm gonna back to Misskey.

Sharkey has a tutorial that I referred to a lot. It's generally great, except that it doesn't really migrate the value of antenna_src_enum. Instead, it reset them to... "home" I think? This specific enum should not be a huge problem, but some other enum might - for example, note_visibility_enum. Without proper value migration, notes meant to be hidden might change to be public, which is apparently not what is desired.

By the way, I was migrating from Firefish 1.0.4-beta2 to Misskey 13.13.0. Different version numbers may lead to different results.

Investigations #

My first part is to compare the database schema between Firefish and Misskey. I have a data dump of Misskey before migrating to Firefish, so that can come in handy for me as a baseline for Misskey's schema.

JetBrains' DataGrip really helped me out in this part. It has a convenient feature to compare the schema of two tables/enums. It can also generate SQL for fairly a few migration types.

Key points #

Simple table with unmatching fields #

Some schema differences are easier to handle. For example:

• There are a lot of tables that have a name beginning with __chart__. They don't seem to be important, so it would be better off to just drop these tables.
• Some columns have a larger size in Misskey (e.g. varchar(256) instead of varchar(128)). They should be able to converted safely without data loss.
• Some apparent column name changes (e.g. meta.ToSUrl to meta.termsOfServiceUrl).
• Some apparent column additions with a default value.
• Some apparent column deletions that are linked to Firefish-exclusive features. They are to be gone by design.

While some are a bit tricky...

Enum migration #

Enumeration definitions in Firefish and Misskey are different at times. In some cases, Firefish has more, while in other cases Misskey has more. Here are some examples that I came across:

• antenna_src_enum   * ('home', 'all', 'users', 'list', 'group', 'instances')
• note_visibility_enum   * ('public', 'home', 'followers', 'specified', 'hidden')
• notification_type_enum   * ('follow', 'mention', 'reply', 'renote', 'quote', 'reaction', 'pollVote', 'pollEnded', 'receiveFollowRequest', 'followRequestAccepted', 'groupInvited', 'app', 'achievementEarned')
• user_profile_mutingnotificationstypes_enum   * ('follow', 'mention', 'reply', 'renote', 'quote', 'reaction', 'pollVote', 'receiveFollowRequest', 'followRequestAccepted', 'groupInvited', 'app', 'pollEnded', 'achievementEarned')

(Items with delete lines exist on Firefish but not Misskey; items with bold style exist on Misskey but not Firefish.)

For enumerations with new variants, we may safely use the old value; for enumerations with fewer variants, instance admins have to find a fallback default value (or, of course, you can delete affected lines altogether). In either case PostgresQL doesn't do the conversion automatically, so several lines of SQL are needed. For example:

BEGIN;
CREATE TYPE public.new_antenna_src_enum AS ENUM ('home', 'all', 'users', 'list');
ALTER TABLE antenna ADD COLUMN new_src public.new_antenna_src_enum;
DELETE FROM antenna WHERE src NOT IN ('home', 'all', 'users', 'list');
UPDATE antenna SET new_src = src::text::new_antenna_src_enum;
ALTER TABLE antenna DROP COLUMN src;
ALTER TABLE antenna RENAME COLUMN new_src TO src;
DROP TYPE public.antenna_src_enum;
ALTER TYPE new_antenna_src_enum RENAME TO antenna_src_enum;
COMMIT;

Redis #

It seems that Misskey and Firefish do only use that as a cache and rate-limiter. I think it can be safely discarded.

Double check 'em #

This section lists some items that you may especially want to check after the database migration and booting Misskey up:

• user_keypair. This table saves the identity key pairs for users on your instance. They should be exactly the same before and after the migration. If for some reason it changed, make sure to get 'em back from your old database dump.
• Passwords. Because Firefish has switched to argon2 while Misskey is still using bcrypt, the Firefish-stored password format cannot be read by Misskey. Because Firefish will do a re-hash to migrate to argon2 (that's a good practice!), the passwords cannot get back to "the Misskey format" after a login. Users on your instances are very likely to be unable to login. Ask them to reset the password.
• Some settings in your meta table. They might need to be changed back as you wish.

Your Mi-leage May Vary! #

Even in the case that everything worked well, there might still be problems while running the "new" Misskey instance. Be prepared to get your hands dirty on SQL to fix the issues!