Experimental report · Evidence through 10 October 2026
Learning, retention, and task-3 unlearning on Tiny ImageNet
The completed three-seed study compares learning task 15 alone with forgetting task 3 or task 14 before learning task 15. Both forgotten tasks stay at 10% accuracy in every seed. Task-15 learning remains possible, but accuracy on other tasks is not uniformly preserved.
Latest results ? 10 October 2026
Completed extension: three seeds and two forgetting targets
Each seed first learns 3 ? 0 ? 9 ? 5 ? 17 ? 1 ? 7 ? 14. We copy its source model into three independent continuations: learn 15 only, forget 3 then learn 15, and forget 14 then learn 15. This tests different request sequences after one source order.
| Seed | Learn 15 only | Forget 3 ? Learn 15 | Forget 14 ? Learn 15 |
|---|---|---|---|
| 0 | 50.8% | 50.8% | 48.2% |
| 1 | 44.0% | 47.2% | 47.8% |
| 2 | 34.4% | 47.4% | 32.6% |
| Sequence | Task-15 changes: seeds 0, 1, 2 | Mean task-15 change | Mean other-task change |
|---|---|---|---|
| Forget 3 ? Learn 15 | +0.0, +3.2, +13.0 | +5.40 | -1.29 |
| Forget 14 ? Learn 15 | -2.6, +3.8, -1.8 | -0.20 | -1.02 |
Changes are percentage points: forgetting branch minus the matching learning-only baseline. Other-task means exclude the requested forgotten task and average seven tasks per seed. Each sequence mean uses three seeds.
Main finding: both forgotten tasks reach 10% accuracy after forgetting and remain there after task-15 learning in every seed. Forgetting task 3 matches or improves task-15 accuracy, but the +5.40-point mean includes a large +13.0-point seed-2 effect. Forgetting task 14 has mixed effects.
Do earlier tasks retain their accuracy?
We evaluate all previously learned tasks except the requested forgotten task. During forgetting, drops are measured against the model before forgetting. After learning task 15, retention is measured against the same seed's learning-only baseline. For forget 3, the retained tasks are 0, 9, 5, 17, 1, 7, and 14. For forget 14, they are 3, 0, 9, 5, 17, 1, and 7.
Five of six forgetting branches have lower mean retained-task accuracy. Individual final losses range up to 14.2 points. Accuracy is therefore not uniformly preserved, even when the requested task stays at chance.
12 jobs, six comparisons, and 39 requests complete. All recorded pairing checks pass. One source order, one incoming task, and three seeds limit generalization. Task identity and learning position change together. No erasure, privacy, or recovery claim is established. The new source models use uniform settings; the earlier pilot below is not included in these averages.
Concise report with charts, settings, and seed-level losses ? Four-slide presentation and interactive experiment tree ? Verified evidence
Presentation summary · Evidence through 10 October 2026
Problem: early runs showed weak new-task learning, falling older-task scores, and very small generated classifier weights.
Initialization decision: check the paper’s Hyperfan method for generated weights. Kaiming stays in the hypernetwork’s hidden layers. This was not a proven fix for forgetting. Why and what changed.
L = learn a task. U = unlearn a task. Each task contains ten image classes; chance accuracy is 10%. Arrows show operation order. All scores are validation accuracy; changes are percentage points.
| Experiment | Sequence or comparison | Key findings |
|---|---|---|
| Protection during new learning | Start: saved model after L3 → L0 → L9 → L5.L17 with beta 0 vs beta 0.1 Beta sets the strength of old-task protection. |
|
| Repeatability | Start: reload that same saved model for each run.L17 twice in fresh GPU sessions Same seed 0 and beta 0.1. |
|
| Successive learning | Start: repeat model after tasks 3, 0, 9, 5, and 17.L1 → L7 → L14 Continue each final model; beta 0.1; no unlearning. |
|
| Unlearning and later learning | Start: same saved model after L14 for both branches.A: L15 B: U3 → L15 Beta 0.1; gamma 0.01; same GPU session. |
|
Limit: these earlier results are one-seed diagnostics and are not pooled with the completed three-seed study above. Both studies cover one later lesson. Chance accuracy does not prove information erasure; recovery and privacy have not been tested.
Settings and architecture · Comparison with the paper · Evidence files. Click an experiment name for its detailed results.
01 · The purpose
Research question and experimental sequence
Our research question is whether useful information remains in a model after deliberate unlearning. Low accuracy shows that the model no longer performs a task well. It does not show whether additional training could recover that task using information still stored in the model.
This report covers the preparation for that recovery question: establishing learning, measuring losses on earlier tasks, and testing deliberate unlearning against a learning-only control. It is an independent diagnostic study of the UnCLe method, not a complete reproduction of its paper.
| Experiment | Question | Use of the result |
|---|---|---|
| Task-17 beta trials and repeat | Does protection reduce old-task losses, and does a repeated run give similar scores? | Select a provisional protection setting and assess the limits of single-run comparisons. |
| Learning tasks 1, 7, and 14 | How do individual older tasks change during several new lessons without unlearning? | Measure ordinary forgetting and save a common starting model for the paired test. |
| Unlearn task 3, then learn task 15 | Does task 3 reach chance accuracy and remain there during later learning? What happens to other tasks? | Establish a measured unlearning result before testing recovery. |
Why we checked Hyperfan initialization
Initialization chooses a model’s numerical values before training. The name is Kaiming, also called He initialization. In a hypernetwork, initializing the generator’s own layers does not by itself specify the starting scale of every classifier weight it generates. Hyperfan provides initialization rules for those generated weights. The UnCLe paper specifies Hyperfan to produce Kaiming-He initialized target parameters. Paper, learning method; Hyperfan reference.
| Problem or check | Observation | Why it mattered |
|---|---|---|
| Learning and retention | Early learning-only runs learned weakly and lost accuracy on earlier tasks. | We needed to review the learning implementation before attributing these losses to deliberate unlearning. |
| Generated classifier weights | A fresh-model CPU check measured a weight spread of 0.0004483 under the old recipe, versus 0.0143448 under the Hyperfan-in option. | The old final-classifier values were much smaller. This justified checking their initialization, but did not prove the cause of weak learning. |
| BatchNorm starting values | The old generated scales were near zero. The revised option starts scale at one and shift at zero. | These parameters control the scale and shift of normalized activations. The revised policy is a documented choice, not a recovered author setting. |
| What changed | We added Hyperfan-in rules for generated weights and revised classifier, bias, and BatchNorm policies. Kaiming initialization remained in the generator’s hidden layers. | This was not a complete Kaiming-to-Hyperfan replacement. The old bias-free convolution formula was already compatible with Hyperfan-in. |
| What the check established | Generated starting-weight statistics and a CPU forward/backward pass were checked before the fresh L3 → L0 → L9 pilot. | These checks support the implementation’s arithmetic. We did not isolate each change in matched training runs, so a causal accuracy improvement remains unproven. |
“Weight spread” above means standard deviation across the generated classifier weights in one fresh model, not variation in final accuracy. Sources: audit and its qualification and saved CPU measurements.
The paired unlearning runner also resolved a difference between its two objectives: both the unlearning penalty and the old-task protection penalty now compare actual generated classifier weights, including scaling and offsets. Earlier unlearning results used a different representation and cannot be treated as results of this corrected runner. The detailed pilot uses a constant learning rate of 0.0001; differences from the paper are listed below.
Historical settings and results: 5–6 October record. The figures below use saved measurements from the later diagnostics. “Older task” means a task learned before the lesson being measured, not an older dataset.
02 · Experimental methods
Dataset, training settings, and neural network architecture
These settings describe the later task-17 diagnostics, learning-only sequence, and paired task-3 unlearning experiment shown in this report. Earlier archived runs had different settings. The architecture below follows the code revision recorded in the paired run, not later uncommitted changes.
Dataset and evaluation protocol
| Item | Setting | Meaning for the experiment |
|---|---|---|
| Dataset | Tiny ImageNet-200, downloaded from the Stanford archive. | 200 classes; 100,000 training images and 10,000 labeled validation images in total. |
| Image format | 64 × 64 pixels, three RGB color channels. | Images enter the classifier at their original size. |
| Task partition | 20 disjoint tasks, ten classes each. Saved class partition generated with seed 42. | The same classes remain assigned to the same task. Partition seed 42 is separate from training seed 0. |
| Images per task | 5,000 training images and 500 validation images. | 500 training images and 50 validation images per class. |
| Preprocessing | Convert to RGB and a tensor with pixel values in [0, 1]. No augmentation or mean/standard-deviation normalization. | No random crops, flips, or pretrained-model preprocessing are used in these runs. |
| Task-aware evaluation | Supply the task identifier; predict among that task’s ten classes. | This is not a single 200-class prediction problem. Expected random-guess accuracy is 10%. |
| Reported accuracy | Top-1 accuracy on the labeled validation split. | One correct class per image. The legacy report key “test” refers to validation; the public test split has no labels. |
Sources: Stanford dataset archive, dataset construction, and saved class partition.
Training and computing settings
| Setting | Value | Scope |
|---|---|---|
| Training seed | 0. | All detailed comparisons here use one seed. |
| Initialization | Hyperfan-in option. No pretrained ImageNet weights. | Inherited from the saved source model. Continuations load its weights rather than initialize a fresh model. |
| Optimizer | Adam, created anew for each learning or unlearning request. | The optimizer changes the trainable hypernetwork values to reduce the chosen loss. |
| Learning rate | 0.0001, constant; no scheduler. | Both learning and paired unlearning use this value. |
| Learning budget | Five epochs, training batch size 64. | 79 batches per epoch, including the last partial batch; 395 updates per lesson. |
| Evaluation batch size | 256. | Evaluation measures accuracy without updating model weights. |
| Learning protection beta | Batch: 0, 0.001, 0.01, 0.1, 1. Repeat, sequence, and pair: 0.1. | Beta multiplies the old-task protection penalty during learning. |
| Paired unlearning budget | 100 updates; ten independent Gaussian noise vectors per update. | Only branch B unlearns task 3. |
| Unlearning gamma | 0.01. | Gamma multiplies the noise loss. The protection term has coefficient one. |
| Recorded environment | Google Colab; NVIDIA A100-SXM4-40GB; Python 3.13.15; PyTorch 2.11.0+cu130; CUDA 13.0. | Recorded in the detailed reports. The repeated runs used different sessions; paired branches shared one session. |
How the hypernetwork produces an image classifier
The hypernetwork generates classifier weights from two short codes. A task code identifies which task the classifier will solve. A chunk code identifies which piece of the classifier’s weights to generate. Joining two 32-number codes gives a 64-number input. A chunk is a group of weight values, not a group of images.
Diagram A. Learning and unlearning
Learning adds a new task using its images and correct labels. The shared generator and the new task code change.
Click a box or choose a part below to read its explanation. On a small screen, scroll the diagram sideways.
Actual classifier weights
These are the final numbers used by ResNet to classify images. The generator outputs are assembled into weights, including the required scales and offsets. Updating the shared generator can change these weights for earlier tasks even when their task codes stay frozen.
| Component | Recorded architecture | Role |
|---|---|---|
| Task code | 32 learned values per task. | Selects a task-specific classifier. |
| Chunk codes | 200 codes of 32 values each, shared across tasks. | Identify pieces of the generated parameters. |
| Shared hypernetwork layers | Fully connected layers: 64 → 128 → 256 → 512, with ReLU after each layer. | Transform codes into features for the output heads. ReLU sets negative values to zero. |
| Output heads | 174 ordinary-weight chunks, two BatchNorm chunks, and 24 residual-projection chunks. | Generate convolution/classifier parameters, BatchNorm scale/shift parameters, and skip-path projection weights. |
| Generated classifier | Modified ResNet50; 23,520,842 parameters per task. | Weights are assembled from chunks, reshaped, scaled, and given the required offsets. |
| Hypernetwork size in the paired report | 134,016,154 parameters, including chunk codes and nine task codes after adding task 15. | This count is separate from the generated classifier parameter count. |
ResNet50 layers used in these runs
ResNet connects convolution blocks with skip paths that add an earlier representation to a later one. A projection changes the skip-path shape when needed. Our target uses Torchvision’s ResNet50 structure with a smaller first convolution and a ten-class output.
| Stage | Structure | Output per image |
|---|---|---|
| Input | RGB image. | 3 channels × 64 × 64 pixels. |
| Stem | 3 × 3 convolution, stride 1, 64 channels; BatchNorm; ReLU; 3 × 3 max-pool with stride 2. | 64 channels × 32 × 32. |
| Residual stage 1 | Three bottleneck blocks. | 256 channels × 32 × 32. |
| Residual stage 2 | Four bottleneck blocks. | 512 channels × 16 × 16. |
| Residual stage 3 | Six bottleneck blocks. | 1,024 channels × 8 × 8. |
| Residual stage 4 | Three bottleneck blocks. | 2,048 channels × 4 × 4. |
| Pooling and classifier | Adaptive average pooling to 1 × 1; flatten; linear layer from 2,048 features to ten scores. | Ten class scores; the highest score gives the prediction. |
A bottleneck block uses 1 × 1, 3 × 3, and 1 × 1 convolutions. The table describes the pinned experiment code, which used the original adaptive average pooling. Sources: recorded target and hypernetwork implementation and Torchvision ResNet source.
What changes during training
| Model state | Learning a new task | Unlearning task 3 in the pair |
|---|---|---|
| Shared hypernetwork layers and output heads | Updated. | Updated. |
| Task codes | New task code updated; earlier codes frozen. | All task codes frozen. |
| Chunk codes | Learned with the first task, then frozen for these continuations. | Frozen. |
| BatchNorm running statistics | Updated only for the task currently being learned. | Frozen for every task. |
| ResNet classifier weights | Generated from the hypernetwork; not independently optimized. | Change through updates to the shared generator. |
BatchNorm has two kinds of values: generated scale/shift parameters, which can change with the hypernetwork, and running means/variances stored separately for each task. Freezing the latter does not freeze all BatchNorm parameters.
Learning and unlearning objectives
Learning loss: classification error + beta × protection.
Paired unlearning loss: gamma × noise distance + protection.
Classification error is cross-entropy, which penalizes incorrect class predictions. Protection sums squared changes in generated weights relative to a frozen copy taken before the request, then averages over protected tasks. Noise distance averages the summed squared distance from task-3 weights to ten fresh Gaussian vectors with mean zero and variance one. Both objectives compare actual generated weights, including scales and offsets.
During unlearning, the seven tasks other than task 3 are protected. During later task-15 learning, all eight earlier tasks are protected, including task 3 at its post-unlearning state in B. Earlier-task training images are not replayed; protection uses generated weights. Gradients pass through the classifier to update the hypernetwork.
Sources: learning and protection code, paired unlearning code, and saved configuration and architecture measurements.
03 · Task-17 repeatability check
Repeat training of task 17 produced a 7-point difference on task 9
Question: if we load the same checkpoint and teach the same new task again, how much do the final scores change? We repeated the beta-0.1 task-17 trial in a fresh GPU session. The starting model already knew tasks 3, 0, 9, and 5. Only task 17 was trained; the four earlier tasks were evaluated.
| Condition | Comparison between runs |
|---|---|
| Starting model and scores | Same checkpoint file hash and same recorded validation scores. |
| Training settings | Seed 0, beta 0.1, learning rate 0.0001, five epochs, and batch size 64 in both runs. |
| Recorded environment | Same code revision and recorded software and GPU details. |
| Session | The second run used a fresh GPU session. These were same-seed repeats, not tests of different seeds. |
Figure 1. Final scores from two beta-0.1 task-17 training runs
Final scores for all five tasks
| Task | First run (%) | Repeat (%) | Repeat minus first (points) |
|---|---|---|---|
| Task 3 | 29.0 | 23.6 | -5.4 |
| Task 0 | 30.8 | 30.8 | +0.0 |
| Task 9 | 49.8 | 42.8 | -7.0 |
| Task 5 | 55.8 | 55.8 | +0.0 |
| Task 17 | 50.8 | 55.8 | +5.0 |
| Finding | Interpretation | Decision |
|---|---|---|
| Task 9: 49.8% → 42.8% | The repeat was seven points lower on an earlier task despite the matching recorded conditions. | Do not interpret small differences between single trials as reliable setting improvements. |
| Task 17: 50.8% → 55.8% | The repeat learned the new task better while retaining some earlier tasks less well. | Assess new-task learning and older-task retention together. |
| Cause of the variation | Unresolved. The epoch reports do not locate the first individual training update where these two runs differed. | Keep beta 0.1 provisional and use a same-session paired comparison for the unlearning question. |
04 · Task-17 protection comparison
Nonzero protection reduced older-task losses during task-17 training
Question: does increasing beta reduce the loss of earlier-task accuracy when the model learns task 17? We compared beta values 0, 0.001, 0.01, 0.1, and 1 from the checkpoint after tasks 3, 0, 9, and 5. Each trial restarted from that checkpoint; one beta trial did not continue another. There was no unlearning request.
Figure 2. New-task learning and older-task retention
New-task accuracy and older-task losses by beta
| Beta | Task 17 (%) | Mean old-task change (points) | Largest old-task drop (points) |
|---|---|---|---|
| 0 | 54.0 | -26.50 | 41.6 |
| 0.001 | 56.8 | -1.55 | 5.8 |
| 0.01* | 52.2 | -0.05 | 3.8 |
| 0.1 | 50.8 | +1.00 | 0.4 |
| 1 | 52.4 | -1.30 | 3.8 |
Without protection, the older-task average fell by 26.5 percentage points. Task 9 fell to 10.0%, and task 5 fell to 13.2%. With beta 0.1, the older-task average rose by 1.0 point and the largest individual drop was 0.4 points, while task 17 reached 50.8%.
The repeat did not reproduce that near-zero individual loss: its older-task average fell by 2.1 points and its largest drop was 5.6 points. We therefore kept beta 0.1 as a provisional setting, without calling it a proven optimum.
Learning and decision: new-task training alone can cause large older-task losses. Nonzero protection reduced these losses in this batch. We used beta 0.1 for the following sequence and pair, but did not establish it as the best value. The beta-0.01 row came from an earlier session, and each beta has only one result here.
05 · Learning-only sequence: tasks 1, 7, and 14
Task 17 lost 5.8 points although the original five-task average was unchanged
Question: do older-task losses accumulate during successive lessons without deliberate unlearning? We loaded the beta-0.1 repeat model after tasks 3, 0, 9, 5, and 17. We trained tasks 1, 7, and 14 in order, keeping beta 0.1. Each lesson started from the preceding lesson’s final model. Task 17 was not trained again.
Figure 3. The original five-task mean and task 17
Validation scores after each completed lesson
| Stage | Task 3 (%) | Task 0 (%) | Task 9 (%) | Task 5 (%) | Task 17 (%) | Task 1 (%) | Task 7 (%) | Task 14 (%) |
|---|---|---|---|---|---|---|---|---|
| Starting model | 23.6 | 30.8 | 42.8 | 55.8 | 55.8 | Not learned | Not learned | Not learned |
| After learning 1 | 25.8 | 27.6 | 47.6 | 56.8 | 55.4 | 54.2 | Not learned | Not learned |
| After learning 7 | 26.2 | 32.6 | 50.6 | 56.4 | 52.4 | 56.8 | 35.2 | Not learned |
| After learning 14 | 27.4 | 31.6 | 45.6 | 54.2 | 50.0 | 49.0 | 31.8 | 47.6 |
Tasks 3, 0, and 9 finished above their sequence-start scores. Task 17 fell after each new lesson. Learning task 14 also lowered task 1 from 56.8% to 49.0%, a 7.8-point loss during that lesson.
| Hypothesis | Observation | Learning |
|---|---|---|
| Losses accumulate across lessons | Task 17 fell after each lesson, from 55.8% to 50.0%. The original five-task mean returned to 41.76%. | Accumulating loss occurred for task 17, not for every original task or their average. |
| The earliest tasks lose most | Tasks 3, 0, and 9 finished above their sequence-start scores. Task 17 declined. | This sequence does not support that simple rule. Task identity and learning order were not varied independently. |
| The average hides individual losses | The unchanged five-task mean included a 5.8-point task-17 loss. | Report individual tasks alongside the fixed-task average. |
| Learning and protection push weights in opposite directions | We sampled these update directions on the first batch of each epoch. Some opposed one another. | The sparse samples did not establish why particular task scores fell. |
Decision: close the learning-only sequence and use its final checkpoint for both branches of the unlearning comparison. This supplies a common model and evidence that ordinary learning can change older-task scores.
06 · Earlier pilot: unlearn task 3, then learn task 15
Earlier one-seed pilot: task 3 stayed at 10% during task-15 training
Historical result from 7 October. The completed three-seed extension is shown at the top of this page; these older scores are not pooled into its averages.
Question: can deliberate unlearning reduce task-3 accuracy to chance, and does that accuracy increase again while learning a new task? Both branches loaded the checkpoint after task 14, containing tasks 3, 0, 9, 5, 17, 1, 7, and 14. The two branches ran in the same GPU session.
A is the learning-only control: it measures changes caused by learning task 15. B adds task-3 unlearning before the same task-15 lesson. Neither branch used older-task training images. Older-task validation images were used only to measure accuracy.
To unlearn task 3, the runner changes its generated classifier weights toward fresh random noise while penalizing changes to the weights generated for the other seven tasks. This operation is intended to reduce task-3 performance. Its success is measured here by validation accuracy, not by a test of information removal.
View matched conditions and training settings
| Check or setting | Recorded result |
|---|---|
| Starting conditions | Starting model, starting scores, and CPU and GPU random states matched. |
| Task-15 training | Initial task-15 code, image order, image values, and update count matched. Both branches completed 395 weight updates. |
| Learning settings | Seed 0, beta 0.1, learning rate 0.0001, five epochs, and batch size 64. |
| Unlearning settings in B | 100 weight updates, ten fresh random noise samples per update, learning rate 0.0001, and gamma 0.01. Gamma multiplies the noise loss during unlearning. |
| Fixed model state | Earlier-task codes, shared chunk codes, and earlier-task BatchNorm running values were unchanged. Task and chunk codes are inputs used by the hypernetwork to generate classifier weights. |
| Unlearning objective | The noise and protection penalties used actual generated classifier weights, including scaling and offsets. |
Figure 4. Task 3 before and during learning task 15
The result we can report: the requested task lost accuracy and did not recover during this one later lesson. Chance accuracy does not show whether useful information remains in the model.
Retained-task results: B averaged 0.34 points below A
Immediately after unlearning, the seven retained tasks had lost 1.54 points on average. At the end of task-15 learning, they finished 0.34 points lower on average in B than A. That small final average difference does not mean every retained task was unaffected.
Figure 5. Final retained-task differences between branches
Task 9 was 5.6 points lower in B, while task 5 was 9.6 points higher. The new task 15 reached 54.4% in A and 49.0% in B. This pair did not show improved new-task learning after unlearning.
Scores before unlearning, after unlearning, and after task-15 training
| Task | Start (%) | B: after unlearning 3 (%) | A: learn 15 (%) | B: unlearn 3, then learn 15 (%) | B minus A (points) |
|---|---|---|---|---|---|
| Task 3 (target) | 27.4 | 10.0 | 35.2 | 10.0 | -25.2 |
| Task 0 | 31.6 | 30.0 | 36.8 | 34.2 | -2.6 |
| Task 9 | 45.6 | 42.4 | 47.0 | 41.4 | -5.6 |
| Task 5 | 54.2 | 53.0 | 46.0 | 55.6 | +9.6 |
| Task 17 | 50.0 | 50.0 | 45.0 | 47.8 | +2.8 |
| Task 1 | 49.0 | 46.8 | 52.0 | 46.6 | -5.4 |
| Task 7 | 31.8 | 30.8 | 29.4 | 30.4 | +1.0 |
| Task 14 | 47.6 | 46.0 | 45.8 | 43.6 | -2.2 |
| Task 15 (new) | Not learned | Not learned | 54.4 | 49.0 | -5.4 |
During unlearning, task 17 temporarily fell from 50.0% to 28.2%
Figure 6. Scores during the 100 unlearning updates
At the first unlearning update, task 17 fell from 50.0% to 28.2%, a temporary 21.8-point loss. Its score recovered to 50.0% by the end of unlearning. A final-only report would miss that disturbance.
Different measures answer different questions
| Measure | What is compared | Result |
|---|---|---|
| Average retained-task change after unlearning | For the seven retained tasks, average their scores after unlearning minus their starting scores. | −1.54 points. |
| Spill, using the paper’s definition | Add the size of each retained task’s score change after unlearning, counting both increases and decreases as positive. | 10.8 points summed across seven tasks. This is not an average. |
| Final retained-task difference between branches | After task-15 learning, average B’s scores minus A’s scores for the same seven tasks. | −0.34 points. |
| Relapse over one later lesson | Task-3 accuracy after learning task 15 minus its accuracy immediately after unlearning. | 0.0 points: 10.0% − 10.0%. |
Spill and relapse definitions: paper, equations 4 and 5. Functional forgetting in this report means low accuracy on the requested task under the current evaluation. It does not mean verified information erasure.
Learning and decision: this pair established low task-3 accuracy through one later lesson. It also showed temporary and task-specific changes on retained tasks, and lower new-task accuracy than the control. We closed this first comparison without another beta or gamma search. The three-seed extension above is now complete. Whether task-3 information can be recovered remains untested.
07 · Interpreting the evidence
Comparison with the UnCLe paper: supported, mixed, and untested claims
The authors report chance-level forget accuracy, low damage to retained tasks, resistance to later relapse, and improved learning in longer sequences. They also report a membership inference test of privacy. Paper, discussion and Tables 1–3.
| Claim | Our assessment | Reason |
|---|---|---|
| Requested task falls to chance | Supported in six extension comparisons | Tasks 3 and 14 reach 10% after forgetting in all three seeds. The earlier pilot also reached 10%. |
| No recovery during later learning | Supported over one later lesson | Both forgotten targets stay at 10% after learning task 15 in every seed, with no recorded evaluation above the 12% threshold. |
| Other tasks remain stable | Mixed evidence | Final mean losses were limited, but individual and temporary drops were larger. |
| Unlearning improves later learning | Depends on request and seed | The extension averages +5.40 points for forget 3 and -0.20 for forget 14. The earlier pilot scored 5.4 points lower. No long-sequence conclusion is established. |
| Privacy under membership inference testing | Not assessed | We did not test whether an attacker could identify which images were used for training. This is the privacy test reported by the paper. |
Differences that limit a direct paper comparison
| Condition | Earlier detailed pair | Paper |
|---|---|---|
| Training seeds | Earlier pilot: seed 0, one pair. The completed extension above uses seeds 0, 1, and 2. | Main results average three seeds. |
| Timing of task-3 unlearning | After learning task 14. | Sequence 1 unlearns task 3 after learning tasks 3 and 0, before task 9. |
| Learning protection beta | 0.1. | 0.01 for Tiny ImageNet. |
| Learning rate | Constant 0.0001. | 0.001 with a scheduler, which changes the learning rate during training. |
| Comparison length | One unlearning request followed by one lesson. | Full sequences of 30 learning and unlearning requests. |
The pair provides limited support for the paper’s functional-forgetting result. It cannot settle the paper’s longer-sequence learning or privacy claims. Some implementation details, including classifier initialization and BatchNorm handling, remain documented reproduction choices rather than verified author settings. Paper, implementation and Appendix C.
Lessons from the completed experiments
- Ordinary learning can lower older-task scores. A learning-only control is necessary when measuring unlearning damage.
- Weight protection helped substantially in the beta trials, but one result cannot establish a stable optimum.
- Individual task scores and intermediate measurements provide information that averages and final scores miss.
- In six matched extension comparisons, the requested forgotten task stays at chance after one subsequent lesson. Other-task accuracy is not uniformly preserved.
Unanswered recovery question: can a small, fixed amount of additional training recover task-3 performance from the unlearned model? A control is needed because training can teach the task again, and information from other tasks can help. Recovery would need to show an advantage attributable to prior task-3 learning. This experiment has not been specified or run. The selected three-seed extension is complete; recovery remains untested.
08 · Sources and scope
Measurement sources and verification limits
The charts were generated from saved JSON measurements in the repository. The later R2 reports were downloaded with file-hash verification. Pair controls were recomputed from the branch reports; remote checkpoint fingerprints were checked. The checkpoint models themselves were not independently loaded during the report review.
- Five-value beta batch
- Beta 0.1 repeat comparison
- Learning sequence: tasks 1, 7, and 14
- Paired U3 and L15 comparison
- Branch A: learning-only report
- Branch B: unlearn-then-learn report
Download the plotted measurements · Source-file fingerprints · Full experiment trajectory
Individual figures: 1, 2, 3, 4, 5, 6. Vector images can be downloaded for presentations.
Reference: Sayanta Adhikari, Vishnuprasadh Kumaravelu, and P. K. Srijith. An Unlearning Framework for Continual Learning. arXiv:2509.17530v1, 2025. Read the paper.
This is the requested public write-up of the recorded experiments. The experiment trajectory remains the detailed record. Model checkpoints and private storage credentials are not included in this page.