Result Notebook¶
This notebook has these sections.
- Brief explanation for the SO lens implementation
- Some overall statistics on all the Neurons
- Analysis for the Tacoma neuron (neuron 1833 of layer 23) and the city neuron (neuron 13769 of layer 23)
- Discussion
I didn't get any meaningful results to put into the medium-depth analysis, so I omitted that section.
Section 0: SO lens Implementation¶
In [ ]:
from modified_llama_final import LlamaForCausalLM
from transformers import AutoTokenizer
import os
import torch
model = LlamaForCausalLM.from_pretrained('meta-llama/Meta-Llama-3-8B', attn_implementation = 'eager').to('cuda')
"""before we compute the SO lens, we first need to do a forward pass"""
tokenizer = AutoTokenizer.from_pretrained('meta-llama/Meta-Llama-3-8B')
text = "The space needle is in"
inputs = tokenizer(text, return_tensors="pt")
inputs = {k: v.to("cuda") for k, v in inputs.items()}
outputs = model(inputs['input_ids'])
"""then we can compute the SO lens using two functions"""
#1.
model.obtain_sec_logits(layer_num = 23 , neuron_num = 1833, pos_from = 3, pos_to = 5)['sec_logits']
#sec_logits is tensor of shape [vocab_size]
#need to first do a forward pass to save the attention weights before applying this function.
#in the forward pass, batch_size = 1
#2.
model.obtain_sec_logits_batched(layer_num = 23 , neuron_num = 1833, pos_from_ls = [3],)
#return tensor of shape [batch_size, seq_len, vocab_size]
#return the SO lens logits for all the positions and all batches where the originating positions are in pos_from_ls.
#need to first do a forward pass to save the attention weights before applying this function.
#in the forward pass, supports batch_size > 1
#More performant than obtain_sec_logits at scale.
"""Lets check that the two functions agree"""
batched_logits = model.obtain_sec_logits_batched(23, 1833, [0,1,2,3])
print(batched_logits.shape)
def logit_len(neuron_num, layer_num = 23, pos_range = [i for i in range(0,4)], pos_to = 5):
total_effect_logits = torch.zeros(model.config.vocab_size).to('cuda')
for pos_from in pos_range:
effect_logits = model.obtain_sec_logits(layer_num, neuron_num, pos_from, pos_to)['sec_logits']
total_effect_logits += effect_logits
return total_effect_logits
diff = batched_logits[0, 5, :] - logit_len(1833)
print(diff.max(), diff.min())
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
torch.Size([1, 6, 128256]) tensor(2.2119e-09, device='cuda:0') tensor(-2.0736e-09, device='cuda:0')
Section 1: Overall Statistics¶
I applied SO lens to the sequence of tokens ['<|begin_of_text|>', 'The', ' space', ' needle', ' is', ' in']. I am interested in the neurons that boost the logit for the token ' Seattle' at position 5. For each neuron, I added up the SO lens logit for token ' Seattle' from position 0 to 5, 1 to 5,..., 5 to 5. The distibution of the SO lens logit for token ' Seattle' is shown below. For comparison, I also included the logit effects for the full ablation of neurons (setting the down projection weight to zero for that neuron and doing a new forward pass). It looks like SO lens are better at picking out key neurons while naive full ablation produces many neurons that seem to have an impact. It is confusing to me how there are neurons whose SO lens create a big change but whose full ablation change is small, and vice versa. I want to look at those neurons and figure our what is going on.
In [ ]:
import torch
import matplotlib.pyplot as plt
import numpy as np
layer23_logits = torch.load('/home/ubuntu/takehome/sec_len_stats/sec_len_statistics_layer_23.pt')
layer23_full_ablation_logits = torch.load('/home/ubuntu/takehome/full_ablation_logit_diff_ls.pt')
layer23_full_ablation_logits = [-x for x in layer23_full_ablation_logits]
# Plot the histogram
num_bins = 100
plt.hist(layer23_full_ablation_logits, bins=num_bins, alpha = 0.7, label='Layer 23 Full Ablation')
plt.hist(layer23_logits, bins=num_bins, alpha=0.7, label='Layer 23 SO lens')
plt.xlabel('Change in Logits for \' Seattle\'')
plt.ylabel('Number of Neurons')
plt.yscale('log')
plt.legend(loc='upper right')
# Show the plot
plt.show()
In [ ]:
import numpy as np
#find the mean and std of the two list
print('Layer 23 Full Ablation mean:', np.mean(layer23_full_ablation_logits))
print('Layer 23 Full Ablation std:', np.std(layer23_full_ablation_logits))
print('Layer 23 SO lens mean:', np.mean(layer23_logits))
print('Layer 23 SO lens std:', np.std(layer23_logits))
plt.scatter(layer23_full_ablation_logits, layer23_logits, alpha=0.3)
plt.xlabel('Layer 23 Full Ablation logit change')
plt.ylabel('Layer 23 SO lens logit change')
Layer 23 Full Ablation mean: 3.617496362754277e-05 Layer 23 Full Ablation std: 0.00201281310407369 Layer 23 SO lens mean: 2.6538214266328447e-05 Layer 23 SO lens std: 0.0018691271270141784
Out[ ]:
Text(0, 0.5, 'Layer 23 SO lens logit change')
The logit value for Seattle was 15.9945. While the SO effect in layer 23 induces an at most 0.2 change to the logit. The full ablation causes a change of similar order of magnitude.
- So I don't think any neuron in layer 23 is vitally important for the fact that the space needle is in Seattle. Although there are certainly neurons that are helping the model to produce the fact such as the Tacoma and the city neuron.
This is not what I expected. But I could not find any problem with my implementation of SO lens (I tried an alternative implementation and checked agreement). In addition, full ablation of the neurons did not cause a much greater change in logit. So I am convinced that no neuron in layer 23 is vitally important.
Next we look at all the layers. Again I am looking at SO lens effect on the token Seattle. Note also that layer 31 has no SO effect by definition.
It is interesting how layer 1 seems to be an outlier. I want to investigate a bit on the neurons that is causing layer 1 to have such high affects on the logit in the future.
In [ ]:
#SO lens across the different layers
mean_ls = []
std_ls = []
max_abs_ls = []
for layer_num in range(32):
so_logits = torch.load(f'/home/ubuntu/takehome/sec_len_stats/sec_len_statistics_layer_{layer_num}.pt')
mean_ls.append(np.mean([abs(x) for x in so_logits]))
std_ls.append(np.std([abs(x) for x in so_logits]))
max_abs_ls.append(max([abs(x) for x in so_logits]))
plt.plot(mean_ls)
plt.xlabel('Layer')
plt.title('Mean for the Absolute Value of SO Lens')
plt.show()
#plot the std around the mean
#plt.fill_between(range(32), [x - y for x, y in zip(mean_ls, std_ls)], [x + y for x, y in zip(mean_ls, std_ls)], alpha=0.3)
plt.plot(std_ls,)
plt.xlabel('Layer')
plt.title('Std for the Absolute Value of SO Lens')
plt.show()
plt.plot(max_abs_ls)
plt.xlabel('Layer')
plt.title('Max Absolute Value of SO Lens')
plt.show()
#plt.ylim(0, 1)
Section 2: Indepth Analysis¶
Before we dive into the the Tacoma and the city neuron, I will discuss how I chose them. To investigate which neuron of layer 23 had the largest effect on the logit for Seattle, I found the top 10 neurons that caused the largest increase and decrease in the Seattle logit. The code below calculate the logit effect on the seattle token for all the neurons in layer23. (It takes a about 6 min to run usually)
In [ ]:
token = ' Seattle'
token_idx = tokenizer.encode(token)[-1]
print(token_idx)
sec_logit_ls = []
for neuron_num in range(model.config.intermediate_size):
sec_logit_ls.append(logit_len(neuron_num)[token_idx].item())
16759
The code below prints out the top 10 neurons that cause the logit of Seattle to increase and decrease. I chose neuron 1833 and neuron 13769 because the logits they boost seem to be interpretable and related to Seattle.
I also found neuron 7298 (Huf, ruf), neuron 10551 (Direction), and neuron 11436 (Golden Garden Park) interesting.
I am really curious about the neurons whose logits do not make immediate sense. I want to filter for top activating context and see if those make sense. I would love to work on using a language model to summarize the top activating context and the top logits as described in the Clarity docs.
In [ ]:
#in case the the previous cell takes too long to run
if len(sec_logit_ls) != len(layer23_logits):
sec_logit_ls = layer23_logits
top10_neuron = torch.topk(torch.tensor(sec_logit_ls), 10)[1].tolist()
lowest10_neuron = torch.topk(torch.tensor(sec_logit_ls), 10, largest = False)[1].tolist()
def top_token_inspection(neuron_num, layer_num = 23, pos_range = [i for i in range(0,6)], pos_to = 5):
print(f'Neuron number {neuron_num}')
#now lets get top logits
total_effect_logits = torch.zeros(model.config.vocab_size).to('cuda')
for pos_from in pos_range:
effect_logits = model.obtain_sec_logits(layer_num, neuron_num, pos_from, pos_to)['sec_logits']
total_effect_logits += effect_logits
print('Effect on the Seattle logit:', total_effect_logits[token_idx].item())
top10 = torch.topk(total_effect_logits, 10)
print('Top 10 tokens and their logits')
for i in range(10):
print('||'+tokenizer.decode([top10.indices[i].item()])+"||", top10.values[i].item())
print('----------------------------------')
print('========Top 10 neurons that have the most increase on the Seattle logit=======')
for neuron_num in top10_neuron:
#print(f'seattle logit shift: {logit_len(neuron_num)[token_idx].item()}')
top_token_inspection(neuron_num)
print('========Top 10 neurons that have the most decrease on the Seattle logit=======')
for neuron_num in lowest10_neuron:
#print(f'seattle logit shift: {logit_len(neuron_num)[token_idx].item()}')
top_token_inspection(neuron_num)
========Top 10 neurons that have the most increase on the Seattle logit======= Neuron number 306 Effect on the Seattle logit: 0.21255050599575043 Top 10 tokens and their logits ||. || 1.5665779113769531 ||----------------------------------------------------------------------|| 1.5435768365859985 ||. || 1.5075844526290894 ||＊|| 1.5039734840393066 ||#ab|| 1.487738013267517 ||---------------------------------------------------------------------- || 1.473102331161499 ||Česk|| 1.4477804899215698 ||.Currency|| 1.4352445602416992 ||#echo|| 1.4177967309951782 ||...... || 1.4152146577835083 ---------------------------------- Neuron number 1833 Effect on the Seattle logit: 0.02260669879615307 Top 10 tokens and their logits || Tacoma|| 0.031041793525218964 || Seattle|| 0.02260669879615307 ||eliac|| 0.021606773138046265 ||别|| 0.021462999284267426 || Pi|| 0.021454187110066414 || Hus|| 0.020824801176786423 || Wa|| 0.020768191665410995 || Pierce|| 0.020641902461647987 || Thai|| 0.020578747615218163 || McGu|| 0.0205382052809 ---------------------------------- Neuron number 7298 Effect on the Seattle logit: 0.01576770655810833 Top 10 tokens and their logits || Hof|| 0.2706979811191559 || Ruf|| 0.25336983799934387 || Laf|| 0.24769288301467896 ||UF|| 0.23665069043636322 || UF|| 0.23408058285713196 ||UFF|| 0.22990451753139496 || Raf|| 0.2293936163187027 || Hoff|| 0.22754019498825073 ||uf|| 0.22065317630767822 || Huff|| 0.21905802190303802 ---------------------------------- Neuron number 13769 Effect on the Seattle logit: 0.015489255078136921 Top 10 tokens and their logits || Seattle|| 0.015489255078136921 || Berlin|| 0.015108034014701843 || Paris|| 0.014330494217574596 || Bangkok|| 0.013882750645279884 || Minneapolis|| 0.01379089429974556 || Tokyo|| 0.013788140378892422 || Madrid|| 0.013641112484037876 || Toronto|| 0.01358332484960556 || Budapest|| 0.013439081609249115 ||Berlin|| 0.013410378247499466 ---------------------------------- Neuron number 11436 Effect on the Seattle logit: 0.014727653004229069 Top 10 tokens and their logits || Golden|| 0.023263579234480858 ||/or|| 0.02240804396569729 ||척|| 0.02222992479801178 || Ed|| 0.021831829100847244 || je|| 0.021552594378590584 ||Ph|| 0.021218523383140564 || Nagar|| 0.02091396413743496 ||alı|| 0.020804688334465027 || Edwin|| 0.020583651959896088 ||ph|| 0.020558517426252365 ---------------------------------- Neuron number 2709 Effect on the Seattle logit: 0.013551173731684685 Top 10 tokens and their logits ||apolis|| 0.03397084400057793 ||uttgart|| 0.03353317826986313 || Hague|| 0.030365273356437683 ||ington|| 0.02998768538236618 ||ortal|| 0.02814902551472187 ||idelberg|| 0.027902647852897644 ||ingham|| 0.02737637236714363 || Marseille|| 0.026533761993050575 || Hann|| 0.025756657123565674 ||furt|| 0.02550688572227955 ---------------------------------- Neuron number 2131 Effect on the Seattle logit: 0.006965428125113249 Top 10 tokens and their logits ||aying|| 0.0456414595246315 ||curring|| 0.04444711655378342 ||pling|| 0.04206839203834534 ||ating|| 0.040837496519088745 ||ressing|| 0.04060714691877365 ||azing|| 0.038848381489515305 ||aping|| 0.03626381978392601 ||aker|| 0.03559019789099693 ||uring|| 0.03520458564162254 ||itting|| 0.034154873341321945 ---------------------------------- Neuron number 10412 Effect on the Seattle logit: 0.006928449496626854 Top 10 tokens and their logits ||nech|| 0.0321979857981205 ||overy|| 0.030882053077220917 ||ucken|| 0.030866898596286774 ||.updateDynamic|| 0.030406812205910683 ||Caption|| 0.02883402816951275 ||'gc|| 0.02874707616865635 ||ií|| 0.028665216639637947 ||yth|| 0.028565770015120506 ||longleftrightarrow|| 0.028311332687735558 ||olian|| 0.027910111472010612 ---------------------------------- Neuron number 1751 Effect on the Seattle logit: 0.006110260728746653 Top 10 tokens and their logits ||rippling|| 0.022204648703336716 ||.Offset|| 0.019482620060443878 ||xae|| 0.019065385684370995 ||?> || 0.018996093422174454 ||_bindings|| 0.018920185044407845 ||nech|| 0.01885603927075863 ||UNCT|| 0.01877909153699875 ||-scrollbar|| 0.01846337504684925 ||iscing|| 0.018419165164232254 ||ní|| 0.018295306712388992 ---------------------------------- Neuron number 13680 Effect on the Seattle logit: 0.005593282636255026 Top 10 tokens and their logits || Bhar|| 0.06004102900624275 || Gujarat|| 0.058878447860479355 || Rajasthan|| 0.058712318539619446 || Kumar|| 0.05869875103235245 || ₹|| 0.0569455660879612 || Karnataka|| 0.056794583797454834 || Gupta|| 0.054365888237953186 || Maharashtra|| 0.0532267689704895 || Rao|| 0.052879542112350464 || Vij|| 0.05238550901412964 ---------------------------------- ========Top 10 neurons that have the most decrease on the Seattle logit======= Neuron number 7708 Effect on the Seattle logit: -0.01724439486861229 Top 10 tokens and their logits ||ackbar|| 0.08373191207647324 ||HandlerContext|| 0.06685570627450943 ||enza|| 0.06277474015951157 ||oire|| 0.062404531985521317 ||... || 0.059722743928432465 ||obe|| 0.05802974849939346 ||ongsTo|| 0.05769483372569084 ||asurer|| 0.05621172860264778 ||ConverterFactory|| 0.0551949068903923 ||913|| 0.054457277059555054 ---------------------------------- Neuron number 10551 Effect on the Seattle logit: -0.006972175557166338 Top 10 tokens and their logits || direction|| 0.046764075756073 ||direction|| 0.04304983839392662 || Direction|| 0.04115739464759827 ||Direction|| 0.03727444261312485 || directions|| 0.03709694743156433 ||-direction|| 0.0352555513381958 ||Directions|| 0.033052295446395874 ||.direction|| 0.03302336484193802 ||irection|| 0.031417086720466614 || Directions|| 0.03138439729809761 ---------------------------------- Neuron number 13240 Effect on the Seattle logit: -0.00660297367721796 Top 10 tokens and their logits ||isti|| 0.02478248067200184 ||irs|| 0.023523613810539246 ||iere|| 0.023281840607523918 || flirt|| 0.023215940222144127 ||lique|| 0.02315097488462925 || || 0.021599125117063522 || Mos|| 0.021445512771606445 || type|| 0.02141604572534561 ||�|| 0.020508484914898872 ||vie|| 0.02049599029123783 ---------------------------------- Neuron number 5013 Effect on the Seattle logit: -0.005727847572416067 Top 10 tokens and their logits || McA|| 0.025672096759080887 || vera|| 0.02403128333389759 ||onta|| 0.02377997152507305 ||lena|| 0.023308217525482178 || UA|| 0.022541452199220657 || INA|| 0.02181573025882244 ||dda|| 0.021638840436935425 ||phia|| 0.021591724827885628 ||ENTA|| 0.021572090685367584 || ASA|| 0.021496426314115524 ---------------------------------- Neuron number 9449 Effect on the Seattle logit: -0.004732874687761068 Top 10 tokens and their logits || Hond|| 0.02046666294336319 ||.omg|| 0.01691846363246441 ||เย|| 0.01678614690899849 || a|| 0.016761450096964836 ||oya|| 0.01650318317115307 || Hudson|| 0.016219014301896095 || an|| 0.01596076972782612 || rest|| 0.015396161936223507 ||Statics|| 0.01449533086270094 ||vore|| 0.01446718629449606 ---------------------------------- Neuron number 8049 Effect on the Seattle logit: -0.004631809424608946 Top 10 tokens and their logits ||oples|| 0.031072447076439857 ||clc|| 0.029976412653923035 ||โด|| 0.029760971665382385 ||ople|| 0.029169198125600815 ||renc|| 0.02868589013814926 ||BorderStyle|| 0.028287794440984726 ||arters|| 0.027939535677433014 ||[--|| 0.02776799164712429 ||borg|| 0.02738591469824314 ||ỷ|| 0.02733006700873375 ---------------------------------- Neuron number 8768 Effect on the Seattle logit: -0.004192182794213295 Top 10 tokens and their logits || Gerr|| 0.015515237115323544 ||ouser|| 0.012520705349743366 ||mare|| 0.012411891482770443 ||dna|| 0.012271683663129807 ||ize|| 0.01218001265078783 ||衡|| 0.012050533667206764 ||śnie|| 0.011963910423219204 ||irical|| 0.01117737591266632 || Truck|| 0.011010434478521347 || /|| 0.0109895970672369 ---------------------------------- Neuron number 6156 Effect on the Seattle logit: -0.004108109511435032 Top 10 tokens and their logits ||.onView|| 0.04680169001221657 ||。。 || 0.0467631034553051 ||jom|| 0.0459529384970665 ||/*************************************************************************** || 0.045888274908065796 || ragaz|| 0.045859139412641525 ||ksen|| 0.04540940746665001 ||oltip|| 0.04435202479362488 ||�a|| 0.044279489666223526 ||'gc|| 0.043463293462991714 || المتحدة|| 0.04313753545284271 ---------------------------------- Neuron number 8646 Effect on the Seattle logit: -0.004004270303994417 Top 10 tokens and their logits ||doi|| 0.017549380660057068 ||ДК|| 0.016417494043707848 ||nty|| 0.016324348747730255 ||ought|| 0.01524318102747202 ||clamation|| 0.015225965529680252 ||very|| 0.015215781517326832 ||ording|| 0.015137056820094585 ||kees|| 0.014954914338886738 || Felipe|| 0.014921673573553562 ||füh|| 0.014870867133140564 ---------------------------------- Neuron number 8336 Effect on the Seattle logit: -0.003838813630864024 Top 10 tokens and their logits || quot|| 0.013688242994248867 ||ived|| 0.013245271518826485 ||ocrates|| 0.012282250449061394 ||atal|| 0.012267385609447956 ||ville|| 0.012212426401674747 ||(newValue|| 0.012071226723492146 ||�|| 0.011997189372777939 ||olid|| 0.011987313628196716 ||, || 0.011928298510611057 || Impress|| 0.01183132454752922 ----------------------------------
City Neuron (13769) I scanned through a portion of fineweb to find example where the neuron value is the highest and the lowest. Below are the top 5 highest and lowest examples. I have denoted the location of highest/lowest activation. Moreover, I print out the top 10 tokens from the SO lens applied to the position of highest/lowest activations.
Interestingly, when the activation is positive, the SO effect boosts the logit for cities, while when the activation is negative, the SO effect boosts the logit of countries and states.
In [ ]:
import random
import numpy as np
from transformers import AutoTokenizer
import torch
top_examples = torch.load('/home/ubuntu/takehome/total_top_examples.pt')
lowest_examples = torch.load('/home/ubuntu/takehome/total_lowest_examples.pt')
tokenizer = AutoTokenizer.from_pretrained('meta-llama/Meta-Llama-3-8B')
def present_example(example, neuron_num):
print('Activation: ', example['activation'])
idx = example['index']
token_ids = example['input_ids'].squeeze().tolist()[:idx+1]
after_token_ids = example['input_ids'].squeeze().tolist()[idx+1:]
print(tokenizer.decode(token_ids, skip_special_tokens=False))
print('##Location for Extreme Activation##\n')
print(tokenizer.decode(after_token_ids, skip_special_tokens=False))
#now we look at the SO lens
model(input_ids = example['input_ids'].to('cuda'))
effect_logits = logit_len(neuron_num, 23, [i for i in range(idx-5, idx+1)], idx+1)
top10 = torch.topk(effect_logits, 10)
print('Top 10 tokens and their logits')
for i in range(10):
print('||'+tokenizer.decode([top10.indices[i].item()])+"||", top10.values[i].item()*10000)
print('----------------------------------')
def inspect_context(neuron):
top_examples[neuron] = sorted(top_examples[neuron], key = lambda x: x['activation'])
lowest_examples[neuron] = sorted(lowest_examples[neuron], key = lambda x: x['activation'])
print('\n====Highest_Activating_examples:====\n')
for examples in top_examples[neuron][-5:]:
print('\n-----new_example------\n')
present_example(examples, neuron)
print('\n===Lowest_Activating_examples:===\n')
for examples in lowest_examples[neuron][:5]:
print('\n-----new_example------\n')
present_example(examples,neuron)
inspect_context(13769)
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
====Highest_Activating_examples:==== -----new_example------ Activation: 6.84375 <|begin_of_text|> beggars cant be choosy but we can make a suggestion. As always, only two things: 1)housekeeping: the knocked at 9am, 2)better coffee. They were a gift so being able to retrieve them was a great surprise. The lobby decor was welcoming and soothing; the concierge were helpful in so many ways from helping one of my friends find their way back home(thank you David) to securing my the (silver hoop) I had lost earlier in the day. We (me and two girlfriends, a true girls weekend away) all agree that our stay was pleasant. The price was a added plus. Trying to stay in downtown ##Location for Extreme Activation## Boston and having to pay for parking is horendous. The main reason we chose the hotel was for the free parking.Snoozing in South Bay This was my 3rd trip to Boston, first time staying in the South Bay area. . They do not give complimentary breakfast but they have breakfast buffet. Apart from that, I enjoyed my stay there. But there was no announcement. Luckily, it was a false alarm. But front desk did not pick up the phone many times and especially when the fire alarm of the whole floor went off at 3am in the morning. Staff was helpful.Good for a short stay The room was clean and comfortable. . So dont ask too many questions, you wont get much help.Nice stay fer a bit Thus hotel is nice if you Top 10 tokens and their logits || Berlin|| 723.9015400409698 ---------------------------------- || Toronto|| 690.4552131891251 ---------------------------------- ||Berlin|| 670.5830246210098 ---------------------------------- || London|| 666.9712066650391 ---------------------------------- || Seattle|| 666.2509590387344 ---------------------------------- ||Toronto|| 649.0794569253922 ---------------------------------- ||London|| 646.8372792005539 ---------------------------------- || Birmingham|| 635.1169943809509 ---------------------------------- || Tokyo|| 632.9891085624695 ---------------------------------- || berlin|| 622.490793466568 ---------------------------------- -----new_example------ Activation: 6.90625 <|begin_of_text|> Rogue Sessions, tickets are $185 and will go on sale a week in advance of each session starting January 2 at giltcity.com ##Location for Extreme Activation## . This includes the 24-course dinner featuring twelve of the visiting chef’s dishes and twelve Rogue 24 favorites with wine and cocktail pairings. There will be 250 seats available per week (50 per night). And Cooper says he’ll be donate a portion of each ticket to Share Our Strength, whose mission is to end childhood hunger in America. In addition to Voltaggio, the roster of chefs includes: José Andrés (Washington, D.C.) Chef/Owner of ThinkFoodGroup and 2011 James Beard Foundation’s Outstanding Chef Tim Byres (SMOKE - Dallas, TX) Chef/Owner of SMOKE Restaurant in Dallas, featured in Food & Wine Magazine as having one of the Ten Best Restaurant Dishes in America (December 2010) Jennifer Carroll (Philadelphia, PA) Chef/Owner of Carroll Couture Cuisine, formerly of 10Arts by Eric Ripert, Le Bernardin, and season 6 of Top Chef John Currence (Oxford, MS) Chef/Owner of City Grocery Restaurant Group, 2009 James Beard Foundation award for Best Chef South, and season 3 of Top Chef Masters Scott Drewno (Washington, D.C.) Chef of Wolfgang Puck’s The Source and the Restaurant Association of Metropolitan Washington’s 2010 Chef of the Year Kats Top 10 tokens and their logits || Seattle|| 187.99027428030968 ---------------------------------- || Houston|| 173.5248789191246 ---------------------------------- || berlin|| 171.74800857901573 ---------------------------------- || Berlin|| 168.40655356645584 ---------------------------------- || Denver|| 167.00077801942825 ---------------------------------- ||Seattle|| 163.00901770591736 ---------------------------------- || Birmingham|| 162.56051138043404 ---------------------------------- || Paris|| 162.04018145799637 ---------------------------------- ||Denver|| 160.7826165854931 ---------------------------------- || Prague|| 159.1019704937935 ---------------------------------- -----new_example------ Activation: 7.71875 <|begin_of_text|> online from India. “It was a bit of a gamble, ordering your wedding gear from the internet but it turned out just perfectly,” says Rebecca. After such careful planning and preparation, the wedding will most likely turn out perfectly too. Rebecca’s little black wedding book: Marquee: LPM Bohemia, lpmbohemia.com Catering: Black Sheep Foods, blacksheepfoods.ie Crockery and catering furniture: caterhire.ie Flowers: Mary O’Gara, flowersmadeeasy.ie Gardening advice and supplies: Scalp Wood Nurseries, scalpwood.com Wedding cake: covecakedesign.ie<|begin_of_text|>hello, I am a fine art photographer and mat my images. I would like to get better at cutting mats (avoid those nasty over cuts) and learn some framing basics, i.e, chopping and connecting Moulding, etc. I use a Logan 850 cutter. I live in Maryland (in metro ##Location for Extreme Activation## dc area). I would like to spend a few hours training with an experienced framer. Of course I will pay for this service.<|begin_of_text|>From Casting Director to Failed Coup Unseating the Turkish military This essay is intended as a follow-up to Phyllis Jeffrey’s earlier piece, “Act One of Turkey’s Post-1980 Political Drama.” The most recent period of competitive democratic politics in Turkey was bookended by two coups: those of 1980 and 2016 Top 10 tokens and their logits || Berlin|| 1074.361801147461 ---------------------------------- || Seattle|| 1015.2240097522736 ---------------------------------- || Toronto|| 1013.5312378406525 ---------------------------------- ||Berlin|| 992.5279021263123 ---------------------------------- ||Toronto|| 952.9337286949158 ---------------------------------- || London|| 949.0855783224106 ---------------------------------- ||Seattle|| 922.7871149778366 ---------------------------------- || berlin|| 920.0210869312286 ---------------------------------- ||London|| 915.8582240343094 ---------------------------------- || Tokyo|| 886.0338479280472 ---------------------------------- -----new_example------ Activation: 7.9375 <|begin_of_text|>That phrase, "The Small World," was perhaps less banal in 1974 than it has become since. I meant it to mean several things. There wass the old phrase, "It's a small world," which is a stock response to an unexpected connection. There was the still older idea of the "microcosm," the small as the model of the great, the individual as the model of the world. There was the contemporary perception that the planet is "shrinking" because of modern connections and the global effect of each small action; there was the recent book title Small Is Beautiful. And finally, the Small World is named for its spiritual begetter (in “Conversation in the Mountains,” Celan names his autobiographical character Klein, or Small, which is the etymological meaning of Paul). In the light of Celan’s work I had seen the world as if miniaturized, and this miniaturization seemed to hold hopes of human destiny becoming manageable, reparable. “Between University and State” was both a concrete location and an intellectual position. In calling attention to the location, I was trying to invoke the city’s vision of itself. ##Location for Extreme Activation## Madison is named for the statesman who is generally regarded as the architect of the United States Constitution. Its Capitol, with its four wings, is a fine example of Neoclassical architecture, a smaller version of the Capitol in Washington. The streets laid out around Capitol Square are named for the signers of Top 10 tokens and their logits ||neapolis|| 473.2971638441086 ---------------------------------- ||ancouver|| 399.6255621314049 ---------------------------------- ||icago|| 385.74468344449997 ---------------------------------- ||stanbul|| 379.50586527585983 ---------------------------------- ||altimore|| 372.2919151186943 ---------------------------------- ||apolis|| 348.23164343833923 ---------------------------------- ||sterdam|| 347.3043069243431 ---------------------------------- ||inburgh|| 345.26098519563675 ---------------------------------- ||tréal|| 343.5058891773224 ---------------------------------- ||amburg|| 338.2106125354767 ---------------------------------- -----new_example------ Activation: 9.0 <|begin_of_text|> the system because it rejects the city's proposed evaluation process. The union insists on more extensive appeals procedures and a sunset clause that would end the system in 2015. If a deal is not reached by midnight tonight, the city will lose the funding. Like other cities, ##Location for Extreme Activation## New York's teachers have long enjoyed greater protection from their employers' processes and their bosses' whims than most American workers. The arguments for extra protection for teachers fall into two categories: appeals to academic freedom and criticism of the evaluation processes. Academic freedom has a terrific ring to it, but it has little to do with today's teachers. The concept goes back to the mid-twentieth century when university professors often faced retaliation for their academic ideas. To a lesser extent, primary and secondary schoolteachers also faced discrimination for their political orientation during the Red Scare, as did many Americans. But in 21st century New York, the rationale is preposterous. New York teachers do not risk dismissal because of their political views any more than the rest of us. Academic freedom is irrelevant. As for the evaluation process, the bugaboo is the Terrible Standardized Test. Test-driven evaluation has inherent flaws that I will not defend. But in the proposed process, test scores account for only twenty percent of a teacher's score. Classroom evaluations account for sixty percent; local considerations account for the last twenty. This process is almost certainly imperfect, but it seems at least reasonable. More importantly, it seems much less arbitrary Top 10 tokens and their logits ||neapolis|| 1834.9719047546387 ---------------------------------- ||ancouver|| 1596.5840220451355 ---------------------------------- || York|| 1422.4319159984589 ---------------------------------- ||olulu|| 1377.2223889827728 ---------------------------------- ||altimore|| 1372.8536665439606 ---------------------------------- || Orleans|| 1372.1509277820587 ---------------------------------- ||stanbul|| 1367.5221800804138 ---------------------------------- || Angeles|| 1337.9721343517303 ---------------------------------- ||inburgh|| 1313.144713640213 ---------------------------------- ||adelphia|| 1299.6046245098114 ---------------------------------- ===Lowest_Activating_examples:=== -----new_example------ Activation: -3.75 <|begin_of_text|>On the other side of the coin, interesting things are happening in the area of the police actually monitoring themselves. There’s been a lot of talk over the last few months about New York City’s “Stop and Frisk” program, which in reality is a “Stop, Talk, and Frisk” program. It was initiated by Rudy Giuliani when he became mayor and is credited with reducing street crime and violent crime to historically low levels. The actual tactic involves police observing behavior, talking to the individual, and frisking them if they have cause, it is not – as Al Sharpton would have you believe – an automatic frisking of every minority the cops can find. I support the policy. The NYPD got taken to court over it and recently a judge ordered the department to put mini-cams on cops to record the encounters. I also support that order. Police have been using cameras for a while and the results are interesting. [A] 2004 study in Criminology and Public Policy by criminologists Stephen Mastrofski from George Mason University and Jonathan Gould from American University evaluated direct observations of police searches in a medium-sized ##Location for Extreme Activation## American city. They conservatively estimated that nearly one-third of police searches were performed unconstitutionally and almost none of those unconstitutional searches came to the attention of the courts. Jay Stanley, a policy analyst with the American Civil Liberties Union (ACLU), calls police-worn video cameras “a win/win for both the public and the police.” Top 10 tokens and their logits || Georgia|| 409.87666696310043 ---------------------------------- || Alabama|| 401.25004947185516 ---------------------------------- || Missouri|| 395.98170667886734 ---------------------------------- || Pennsylvania|| 393.3568671345711 ---------------------------------- || Ohio|| 386.9449347257614 ---------------------------------- || Rhode|| 384.99046117067337 ---------------------------------- || Illinois|| 381.19252771139145 ---------------------------------- || Albania|| 370.10665982961655 ---------------------------------- ||ennessee|| 369.2856803536415 ---------------------------------- || Tennessee|| 356.46598786115646 ---------------------------------- -----new_example------ Activation: -3.609375 <|begin_of_text|>.<|begin_of_text|>FBI Director James Comey, who was rebuffed by the White House after suggesting that a “Ferguson Effect” could be behind the latest uptick in crime, may have found at least some vindication from a recent study posted on the American Psychological Association’s website, entitled “The Alleged ‘Ferguson Effect’ and Police Willingness to Engage in Community Partnership.” The study is paywalled, but Tom Jacobs of the Pacific Standard has a good summary: A new study focusing on one mid-sized ##Location for Extreme Activation## sheriff’s department found negative publicity about policing has indeed reduced some deputies’ motivation, as well as their willingness to form partnerships with members of their communities. […] In February, the researchers surveyed 567 deputies at a mid-sized sheriff’s department in the Southeastern United States (an impressive 85 percent response rate). The deputies were asked a series of questions regarding how recent negative publicity about police actions has impacted their work. They indicated, on a five-point scale, the extent to which such coverage has “made it more difficult for you to be motivated at work,” and “caused you to be less proactive on the job.” […] “It appears that officers in our sample have been affected by negative ‘Ferguson-type’ press,” [the authors] write. “Some officers indicated being less motivated to perform their duties. This is important from a managerial standpoint, because feelings such as these need to be subverted if possible.” The study does not determine—or attempt to determine— Top 10 tokens and their logits || Georgia|| 147.85850420594215 ---------------------------------- ||ennessee|| 130.14604337513447 ---------------------------------- || Tennessee|| 124.81828220188618 ---------------------------------- || Kentucky|| 122.5921418517828 ---------------------------------- || Virginia|| 121.56371027231216 ---------------------------------- ||igeria|| 119.25577186048031 ---------------------------------- || Arizona|| 118.33716183900833 ---------------------------------- ||Georgia|| 117.12738312780857 ---------------------------------- || Oregon|| 115.84674008190632 ---------------------------------- || Italy|| 115.57747609913349 ---------------------------------- -----new_example------ Activation: -3.421875 <|begin_of_text|> stipend to compensate them for being forced to live in the city. They further demanded no fewer than five employees on every piece of fire apparatus whenever temperatures dip below 20 degrees or rise above 90. The same five-employee minimum would have applied when the Department of Homeland Security raises the nation's threat level to red. City Hall also would have been prohibited from summoning outside fire or emergency medical services until "all ancillary equipment" and rehiring available from within the Chicago Fire Department had been exhausted. The last time Local 2 negotiated a new four-year deal, firefighters gave up so much in exchange for their 16 percent pay raise that the backlash ultimately cost then-union President Jim McNally his job. The share of promotions based on merit was increased from 10 per-cent to 16 percent. Chicago became the nation's first major ##Location for Extreme Activation## city to test randomly for both alcohol and drug use, a key provision given the notorious antics in some firehouses over the years. City Hall pegged the cost of the police contract at $385 million and borrowed $160 million to bankroll the back pay. The cost of the firefighters contract was not known. Nor was it clear whether the city would need a similar short-term borrowing.<|begin_of_text|>David McKay Movies and Career Information David McKay is a Scottish actor and television director. In the television series Shoebox Zoo, he plays McTaggart. McTaggart is the Keeper of the Book of Forbidden Knowledge who has been bound to stay with Michael Top 10 tokens and their logits || Georgia|| 303.993858397007 ---------------------------------- || Tennessee|| 266.7303755879402 ---------------------------------- || Pennsylvania|| 262.5875920057297 ---------------------------------- || Virginia|| 259.35545563697815 ---------------------------------- ||ennessee|| 256.02560490369797 ---------------------------------- ||Georgia|| 255.28863072395325 ---------------------------------- || Alabama|| 254.55020368099213 ---------------------------------- || Missouri|| 249.7629076242447 ---------------------------------- || Illinois|| 248.26327338814735 ---------------------------------- || Rhode|| 246.56429886817932 ---------------------------------- -----new_example------ Activation: -3.359375 <|begin_of_text|> rights reserved.<|begin_of_text|>SF police chief pushes for Taser use by officers SAN FRANCISCO San Francisco Police Chief George Gascon said he wants his officers to be armed with stun guns so they have a less lethal option when dealing with aggressive suspects. San Francisco is one of the few major ##Location for Extreme Activation## U.S. cities that does not arm its officers with Tasers. At a news conference Wednesday, Gascon urged the city's police commission to approve the use of the stun guns. Commission members rejected the idea last week although it plans to take it up again next Wednesday. A department study found that one-third of officer-involved shootings over five years might have been avoided by using stun guns. "This is a very critical issue for this community," Gascon said. "It is inexcusable, it's negligent for us not to have the ability to equip our police officers with Tasers." Gascon said arming officers with stun guns would reduce the number of police injuries, save some suspects' lives and save the city money in litigation. He said the Tasers are the "right tool at the right time," and the department will create a policy that allows officers to use Tasers only with aggressive, combative suspects. Some police commission members have worried about the safety of Tasers and said they needed more time to research incidents where suspects have died after being stunned. Three commissioners were in attendance on Wednesday. One of them, Jim Hammer, who voted against the Taser proposal last week said he Top 10 tokens and their logits || Pennsylvania|| 1022.5837677717209 ---------------------------------- || Georgia|| 999.8724609613419 ---------------------------------- || Kentucky|| 950.4522383213043 ---------------------------------- || Virginia|| 914.59721326828 ---------------------------------- || California|| 909.5674753189087 ---------------------------------- || Florida|| 905.5835008621216 ---------------------------------- || Massachusetts|| 893.5391902923584 ---------------------------------- || Alabama|| 889.0216052532196 ---------------------------------- || Vermont|| 877.3775398731232 ---------------------------------- || Ireland|| 874.8269826173782 ---------------------------------- -----new_example------ Activation: -3.1875 <|begin_of_text|> door and gave her aid. The suspect, Michael Wayne Lester, fired more rounds at arriving officers. He was eventually arrested in his house by a SWAT team. Inside, they found over a thousand rounds of ammunition, the SKS, a Ruger Mini-30, and a shotgun. Ofc. Booth remains in critical, but stable condition. She was wearing a ballistic vest, though it would not have stopped the rifle round. She was nevertheless struck below the vest’s front panel. Lester is charged with 4 counts of attempted murder of a law enforcement officer, possessing a firearm while engaged in a felony, shooting in an occupied dwelling, aggravated battery, and resisting arrest with violence. I have noticed a dismally low number of officers carrying a secondary handgun in my area. I have implored our employees to take advantage of the ability to carry a back-up, since our department policy did not allow it until several years ago. The secondary firearm may be needed when you are not able to access your primary handgun or in the event your primary fails. I do not know if Ofc. Booth had a secondary firearm, but I hope she did. Please stay prepared and alert when you go into the field. Prayers to Ofc. Booth for a complete recovery. Randall is a twenty-three year sworn police officer in a mid-sized ##Location for Extreme Activation## Florida police department. He has been an FTO, K9 Handler, Detective, and SWAT Team Leader. He is currently the Midnight Top 10 tokens and their logits || Georgia|| 223.7648330628872 ---------------------------------- ||ennessee|| 217.40956231951714 ---------------------------------- || Missouri|| 211.06529980897903 ---------------------------------- || Rhode|| 209.54156294465065 ---------------------------------- || Peru|| 207.4688859283924 ---------------------------------- || Ohio|| 203.74558866024017 ---------------------------------- ||shire|| 203.24643701314926 ---------------------------------- || Alabama|| 202.12505012750626 ---------------------------------- || Lithuania|| 199.79676231741905 ---------------------------------- || Albania|| 198.32106307148933 ----------------------------------
Tacoma Neuron (1833) Interestingly, the neuron has high activation when something in Washington state that is not Tacoma is mentioned. It has low activation when Tacoma itself is mentioned.
The SO len logit output did not make as much sense as the city neuron. It is probably because the positions where I am applying the SO lens is not the most optimal. In the original VIT setting, it is clear which position to apply the lens to, but for LLM, this is less clear. I would love to investigate whether we can use the attention weights to figure out which pair of positions will create logits that are more interpretable
In [ ]:
inspect_context(1833)
====Highest_Activating_examples:==== -----new_example------ Activation: 7.9375 <|begin_of_text|> believe in investing in LWSD Schools. With our schools reaching crisis overcrowding, we are placing our children’s educational opportunities at risk. King 5 reported April 15 students at LWSD’s Rockwell Elementary overcrowding is so severe (200 kids over capacity) that classes are being taught in portables, divided classrooms and, yes, even in the hallways. To date, our teachers and principals have responded like heroes to the crisis. However, even their efforts can’t magically make room for another 4,000 kids. As Rockwell Principal Kirsten McArdle's told King 5: "We really have run out of options. Every space we have in this school is being used." What will an extra $10-11 a month buy our children … and the kids soon to arrive … with this bond? • Three new elementary schools (two in Redmond and one in ##Location for Extreme Activation## Kirkland – where the overcrowding is most severe and the most growth is expected); • Rebuild and expand Juanita High School (originally built as an “school without walls” that had to be rehabbed with walls and ventilation); • Build a new STEM-focused high school (giving westside students the same great learning experience of LWSD’s student at first STEM high school on the eastside); and • Expand Lake Washington High School (built before ninth-grade students were shifted into high schools). If the minority of the districts voters say no to better educational opportunities to today’s students Top 10 tokens and their logits ||儀|| 31.53576049953699 ---------------------------------- ||ote|| 24.231616407632828 ---------------------------------- || Margin|| 23.396620526909828 ---------------------------------- ||ota|| 22.942228242754936 ---------------------------------- ||VOID|| 22.8897575289011 ---------------------------------- ||agas|| 22.543997038155794 ---------------------------------- ||STM|| 22.435528226196766 ---------------------------------- || Borg|| 22.290614433586597 ---------------------------------- ||ไฟ|| 22.163528483361006 ---------------------------------- || margin|| 22.156438790261745 ---------------------------------- -----new_example------ Activation: 7.96875 <|begin_of_text|>ace, the franchise's previous best opening. Read more at NZ Herald<|begin_of_text|>- About Us - Local Savings - Green Editions - Legal Notices - Weekly Ads They’re not just selling cookies — Girl Scouts also building bridges, robots Most people tend to associate Girl Scouts with cookies rather than science, technology, engineering and mathematics (STEM), but members of two local troops are doing their part to change this. From building bridges to building robots, these young ladies are proving science can be fun. BRIDGING THE SCIENCE GAP Maggie Duffy, a Redmond High School (RHS) senior and member of Girl Scout Troop 43266, is organizing and leading two mechanical engineering workshops for middle school students. The two-day workshops will be held at Evergreen Middle School (EMS) and Redmond Middle School (RMS). The workshop at EMS, 6900 208th Ave. N.E., will be held from 3-5 p.m. on Feb. ##Location for Extreme Activation## 1 and 22. The workshop at RMS, 10055 166th Ave. N.E., will be held from 11:30 a.m. to 1:30 p.m. on Feb. 2 and 23. Aimed toward sixth- through eighth-graders, the workshop is $20. To register, visit www.redmond.gov/register and enter code 44329 for the EMS workshop or code 44330 for the RMS workshop. Top 10 tokens and their logits ||ours|| 57.65319801867008 ---------------------------------- ||imus|| 56.00640084594488 ---------------------------------- || ifndef|| 55.697401985526085 ---------------------------------- ||contres|| 55.225263349711895 ---------------------------------- ||anine|| 55.2100595086813 ---------------------------------- || Leban|| 54.55591715872288 ---------------------------------- ||exus|| 54.09320816397667 ---------------------------------- ||ousand|| 53.08988969773054 ---------------------------------- || esac|| 52.81799007207155 ---------------------------------- ||oves|| 52.73882299661636 ---------------------------------- -----new_example------ Activation: 8.375 <|begin_of_text|>'s Northlake News. Maltby to build biggest snow cone And it's going to be edible. W-D widening may be studied Until now, the county wasn't going to look at the Woodinville-Duvall Road until the year 2003. Summer fun in Bothell There is something for everyone. Check it out. Sheriff looking for volunteers "In a nutshell, we're doing the survey to get people's impression of crime and the job we're doing out there," said Joe Lewis of the Sheriff's Research, Planning and Information unit. County treasurer elected to state association Snohomish County Treasurer Dantini, who already serves on several state committees, was elected to this new post. Summer fun in ##Location for Extreme Activation## Woodinville Art, skating, volleyball, baseball....are you interested? Engagement: Zaikoski - Krisman Following their wedding, this couple will reside in Norfolk, Virginia. New Eagle Scout honored This scout's Eagle project was to organize and coordinate the building of two benches overlooking the playground at Kenmore Park. Group challenges residents to plan car-free getaways Plan a car-free getaway and get a free train ticket, a camera, lattes and more. Showbiz - volunteers needed Work is underway for a 1999 Woodinville community variety show.<|begin_of_text|>|SURFACE TRANSPORTATION BOARD DECISION DOCUMENT| |CSX TRANSPORTATION, INC.--ABANDONMENT EXEMPTION Top 10 tokens and their logits ||儀|| 55.94116635620594 ---------------------------------- ||atz|| 44.16905343532562 ---------------------------------- || président|| 43.396977707743645 ---------------------------------- || Kemp|| 42.21175331622362 ---------------------------------- ||iske|| 41.6236137971282 ---------------------------------- ||ascal|| 40.88889341801405 ---------------------------------- ||ogi|| 40.32615106552839 ---------------------------------- ||定|| 39.778524078428745 ---------------------------------- || corn|| 39.393012411892414 ---------------------------------- || soft|| 39.10894971340895 ---------------------------------- -----new_example------ Activation: 9.25 <|begin_of_text|> model. "It's going to be free-to-play - it'll have some twists, but that's the easiest way for people to think about it," Newell said. He didn't offer specifics as to exactly how it will work when the game launches, but hinted that players who are "valuable" to the community might get some kind of special treatment. "The issue that we're struggling with quite a bit is something I've kind of talked about before, which is how do you properly value people's contributions to a community? "We're trying to figure out ways so that people who are more valuable to everybody else [are] recognised and accommodated. We all know people where if they're playing we want to play, and there are other people where if they're playing we would [rather] be on the other side of the planet. "It's just a question of coming up with mechanisms that recognise and reward people who are doing things that are valuable to other groups of people." What do you think of DotA 2's "F2P" model? Like it or not? And what kind of model will it be? Maybe f2p with in-game shops?<|begin_of_text|>Accident witnesses sought On Wednesday, Oct. 2, at approximately 4:50 p.m., at the intersection of Woodinville-Duvall Rd. and ##Location for Extreme Activation## 156th Ave. NE, across from AM/PM, a 1985 blue Ford Tempo was struck by a 199 Top 10 tokens and their logits ||ours|| 264.97531682252884 ---------------------------------- || Jr|| 219.6863293647766 ---------------------------------- ||IGHT|| 213.83654326200485 ---------------------------------- ||лагод|| 211.1007645726204 ---------------------------------- || jr|| 207.60523155331612 ---------------------------------- ||apel|| 205.05473017692566 ---------------------------------- ||량|| 204.6407014131546 ---------------------------------- ||ày|| 200.533214956522 ---------------------------------- ||jectories|| 199.65549930930138 ---------------------------------- ||conn|| 198.67904484272003 ---------------------------------- -----new_example------ Activation: 9.375 <|begin_of_text|> well as last name with a disturbing number of double consonents. That being said, there's one project on his credits page which has us most interested. Resident Evil 6. Very quickly we scoped out Joe's IMDB page, assuming he'd simply typed the wrong number. Yet the page makes no mention of any other Resident Evil appearances. Has the latest in Capcom's survival horror franchise been outed by a voice actor? Uh oh Joe, you could be in for some trouble...<|begin_of_text|>- A new connection from 196th/195th Avenue NE to NE Novelty Hill Road and rebuilding 196th Avenue NE from NE 95th Street to NE Union Hill Road. - Two roundabouts on 196th Avenue NE at Novelty Hill Road and NE Union Hill Road - Widening NE Union Hill Road to four lanes from ##Location for Extreme Activation## 196th Avenue NE west to the Redmond city limits and widening Evans Creek Bridge to four lanes. - Providing shoulders or sidewalks with bicycle lanes along all roads in the project area and raising Evans Creek Bridge to allow for future extension of the Bear/Evans Trail. - Creating three acres of wetland and improving stream habitat and crossings for fish and wildlife. For further updates on construction and the project, you are invited to sign up to receive emails or test messages at the project website, www.kingcounty.gov/noveltyhillroad (follow the link at top right to Traffic impacts/detours). Construction of Phase One of King County's Top 10 tokens and their logits ||ours|| 382.89304822683334 ---------------------------------- ||avou|| 372.65900522470474 ---------------------------------- ||лагод|| 370.20523101091385 ---------------------------------- ||rieve|| 366.56729876995087 ---------------------------------- || jr|| 364.9626299738884 ---------------------------------- || Jr|| 361.2084314227104 ---------------------------------- ||IENCE|| 352.0379588007927 ---------------------------------- || Sayı|| 351.3883799314499 ---------------------------------- ||importe|| 346.96072340011597 ---------------------------------- ||로나|| 344.8755294084549 ---------------------------------- ===Lowest_Activating_examples:=== -----new_example------ Activation: -11.25 <|begin_of_text|> face. "I really believe in all my heart and all my soul that RJ is a good person and he's a good kid and that one mistake that dreadful night should not ruin the rest of his life." Minutes earlier, Lewis, 19, pleaded guilty to counts of vehicular homicide and one count of attempting to elude a pursuing police vehicle in the July 31 crash in Tacoma that killed his friends, James Paggeot and Walter Corey. Prosecutors said Lewis was fleeing police dispatched to a convenience story where Paggeot and Corey had just stolen beer when he lost control of his car and crashed near ##Location for Extreme Activation## 64th and McKinley on Tacoma's East Side. Paggeot, 22, and Corey, 21, died at the scene. Lewis injured his left arm in the crash and ran away. He surrendered to police a few hours later. Lewis originally was charged with five felonies, but deputy prosecutor John Macejunas dropped two charges after Lewis agreed to take responsibility for his actions. Macejunas and defense attorney Derek Smith recommended a low-end prison sentence of three years for Lewis, which Judge Bryan Chushcoff imposed. The high end was four years. The hearing was highly emotional, with nearly 100 people cramming into a small courtroom to witness it. Almost all were there to support Lewis, including dozens of young men and women who cried and hugged throughout the proceeding. Corey's sister, Sarah Corey, also addressed Chuschcoff Top 10 tokens and their logits || to|| 349.99266266822815 ---------------------------------- || when|| 325.4961967468262 ---------------------------------- || because|| 313.05599957704544 ---------------------------------- || during|| 311.0582008957863 ---------------------------------- || =|| 289.1829237341881 ---------------------------------- ||1|| 277.96508744359016 ---------------------------------- || since|| 241.07865989208221 ---------------------------------- || “|| 240.98996073007584 ---------------------------------- || due|| 227.80196741223335 ---------------------------------- ||3|| 227.67452523112297 ---------------------------------- -----new_example------ Activation: -11.1875 <|begin_of_text|> and owner of Paradise Bowl explains, "today's youth have a wide range of activities to choose, many... - July 31, 2015 - Paradise Bowl Classic Accessories Launches Dynamic Product Finders to Make it Easy to Find the Perfect Fit. - July 16, 2015 - Classic Accessories Metro Auto Rebuild South is becoming Fix Auto Tacoma South; since 1997, Metro Auto Rebuild South has serviced the local residents of ##Location for Extreme Activation## Tacoma with all their auto body and collision needs. Now becoming part of a larger network, Metro moves into the Fix Auto network with all the same ownership, staff, and same great service. - April 28, 2015 - Fix Auto Tacoma South With the release of “Cairo Café” this month, a new comic book series set in the same world as parent book series “Northwest Passage Mysteries,” creator and writer Gary Alan Hidalgo has accomplished the first phase of his transmedia vision of a shared storytelling universe... - March 11, 2015 - Gary Alan Hidalgo Sarah Palin keynotes MOWP 4th annual fundraiser hosted by WoodsEdge Community Church in The Woodlands, Texas. - November 11, 2014 - Mighty Oaks Warrior Program "The Promiscuous Puppeteer," by bestselling author Walter B. Biondi was fully released by Tate Publishing, LLC, on August the 12th for national and international distribution to all traditional book markets. - August Top 10 tokens and their logits || T|| 505.6050792336464 ---------------------------------- || Y|| 468.2542383670807 ---------------------------------- || Tacoma|| 458.7068781256676 ---------------------------------- ||-T|| 451.3438791036606 ---------------------------------- || TS|| 438.24315071105957 ---------------------------------- || TT|| 437.1662437915802 ---------------------------------- || Т|| 430.85888028144836 ---------------------------------- || Q|| 417.76206344366074 ---------------------------------- || Ts|| 416.9800877571106 ---------------------------------- ||/T|| 411.88817471265793 ---------------------------------- -----new_example------ Activation: -11.125 <|begin_of_text|> each of these stations. Explore them all, or find one that best fits your needs! You'll find that attractions are sorted by walking time and distance from the stop, and whether you'll need to head up one of Tacoma's famous hills. Tacoma Link Light Rail Quick Look - Trains run every 12 minutes, except for the early mornings and evenings. Service begins at 5:00am and runs until 10:00pm. Check out the schedule for more details. - Thirteen stops connecting Tacoma's historic Hilltop and Stadium districts to the Tacoma Dome transit station. - Operates 365 days-a-year. - It does NOT connect with Seattle's light rail. At least, not yet. South Sound Craft Crawl Craft beer lovers are in luck! The South Sound Craft Crawl highlights local craft breweries in ##Location for Extreme Activation## Pierce County and the South Puget Sound. Visit enough breweries, collect enough stamps, and redeem for a one-of-a-kind souvenir. Eight of these breweries are located along the Light Rail, making it easy to find craft beer no matter where you step off the Link. Get off the Light Rail here for historic buildings and a bustling dining and nightlife scene. Attendees looking for post-meeting drinks will find plenty here, from flaming tropical inspired drinks to a casual but deliciously local craft brew. This stop is the closest to the Greater Tacoma Convention Center, Tacoma's Visitor Information Center , and several downtown hotels and meeting spaces, making the gateway from Top 10 tokens and their logits ||quence|| 161.47786751389503 ---------------------------------- || anale|| 154.718067497015 ---------------------------------- ||formace|| 137.46270909905434 ---------------------------------- ||ITTE|| 136.59900054335594 ---------------------------------- ||anson|| 134.19653289020061 ---------------------------------- ||igma|| 129.97829355299473 ---------------------------------- ||STANCE|| 129.1678473353386 ---------------------------------- ||layan|| 128.6644395440817 ---------------------------------- ||htags|| 128.54169122874737 ---------------------------------- ||lém|| 125.66214427351952 ---------------------------------- -----new_example------ Activation: -10.8125 <|begin_of_text|> that.” Frandsen, winner of December’s Seattle Marathon, almost didn’t run Tacoma City. The Spokane native was training to run the May 10 Eugene Marathon, but changed his mind when he learned the race was on Mother’s Day. With a 1-month-old and 2-year-old at home, Frandsen said, “My wife deserves a Mother’s Day.” So he changed plans and entered the Tacoma City Marathon. And he’s glad he did. “I think this is a great course,” Frandsen, 37, said. “I like a good, rolling course. The bridge reminds me of doing the Portland Marathon. I really like going through the (Point Defiance) park area. … And I think my favorite part was along ##Location for Extreme Activation## the waterfront.” The previous race record was 2:38:11 by Tacoma’s Michael Lynnes in 2008 when the race used a different route. Frandsen’s record pace wasn’t as fast as he’d hoped. He wanted to finish in less than 2½ hours. “But I haven’t been getting enough sleep lately (because of the baby),” Frandsen said. Jorgensen, the cross-country coach at Life Christian Academy, also has a newborn who has taken a chunk out of his sleeping schedule. He took the lead at the start and won by nearly a minute. Jorgensen says he prays while he runs. “Faith is a big part of my running,” Top 10 tokens and their logits || Tau|| 394.20895278453827 ---------------------------------- || Pierce|| 379.1414946317673 ---------------------------------- || TPM|| 355.96851259469986 ---------------------------------- || Ellis|| 353.8811579346657 ---------------------------------- || Tacoma|| 347.9105234146118 ---------------------------------- || Thai|| 347.9072079062462 ---------------------------------- || same|| 347.04186022281647 ---------------------------------- || Same|| 346.73046320676804 ---------------------------------- || Sum|| 342.5033390522003 ---------------------------------- || Tan|| 336.28854900598526 ---------------------------------- -----new_example------ Activation: -10.75 <|begin_of_text|>ic Want to be treated like royalty? Head to the Crown Bar. Loosely inspired by the Crown Bar in Belfast Northern Ireland, Tacoma’s Crown Bar is nestled in ##Location for Extreme Activation## the heart of the 6th Avenue business district. It is flanked on either side by House of Tattoo and Retrospect Vintage Clothing. Its warm and cozy atmosphere is inviting, the wait staff, welcoming. If you’ve never noticed it, it’s because of the size. It’s pretty miniscule. However, that makes it the perfect venue for an intimate date, a night out with a couple friends, or a happy hour stop on the way home from work. Speaking of happy hours, Crown Bar has one of the best. It runs every day from 4 pm to 6 pm, plus 10 pm to close. Happy hour includes lots of food and drink specials. Wednesday night they offer half price bottles of wine and 40 cent wings. Thursday is taco night with $4 tacos, $3 margaritas and $2 Tecate cans. The regular menu consists of American as well as international dishes – something for everyone. They strive to work with local farmers and ranchers in order to make you fresh, eclectic meals. Starting on March 12 they will be serving brunch Sat-Sun 9am-3pm. The inside will remind you of a rustic and homey pub, including wood fixtures, atmospheric lighting, quaint wall hangings and corner fireplace. A downside is lack Top 10 tokens and their logits || TP|| 828.2576501369476 ---------------------------------- || Tau|| 821.7860013246536 ---------------------------------- || TPM|| 798.024907708168 ---------------------------------- || Pierce|| 792.715847492218 ---------------------------------- || Tacoma|| 781.4562320709229 ---------------------------------- || Res|| 764.1401141881943 ---------------------------------- || TG|| 759.8726451396942 ---------------------------------- || Thai|| 741.8268173933029 ---------------------------------- || Tor|| 740.5311614274979 ---------------------------------- || N|| 733.6521148681641 ----------------------------------
Golden Garden Parks neuron (11436)
In [ ]:
inspect_context(11436)
====Highest_Activating_examples:==== -----new_example------ Activation: 10.5 <|begin_of_text|>! I was late to church that day and the pastor was halfway through his sermon. Thankfully he had a slide that referenced the seven signs of the messiah and so I quick decided on the layout of the sketchnote. Took down the main points in rough pencil and got back to sketch the entire thing again. Am quite pleased with this one, lots of reminders and proper referencing to various verses in the bible. Thanks for sharing with us, Claire!<|begin_of_text|>Maintain your vehicle by staying on top of all recommended services. Choose a service below to get started! We provide complete automotive repair and regular maintenance services for personal vehicles and entire fleets. Our ASE-certified technicians fix vehicle problems right the first time, using top quality parts. See us for everything from an A/C repair to transmission repair. We also provide Diesel Engine Repair and Hybrid Repair. 15th Street Automotive has been your local auto repair shop and fleet service center in Puyallup since 2002. We're a AAA-Approved Repair Facility and a Federated Car Care Center serving Puyallup, Graham, Sumner, ##Location for Extreme Activation## WA, and surrounding areas. We’re the Puyallup area’s #1 choice for customer service and satisfaction. We procure the best parts available and continuously train to make sure the job is done right the first time. <-------- This product is not currently licensed. Please license this product by clicking here. --------><|begin_of_text|>One third of articles in Australia’s major newspapers rejected or cast Top 10 tokens and their logits || Sil|| 736.6807013750076 ---------------------------------- || Tony|| 728.6739349365234 ---------------------------------- || Bart|| 723.9720225334167 ---------------------------------- || Mann|| 698.3640044927597 ---------------------------------- || Han|| 697.513073682785 ---------------------------------- || sil|| 689.2137229442596 ---------------------------------- ||Tony|| 680.0772249698639 ---------------------------------- || Hart|| 664.3891334533691 ---------------------------------- ||Han|| 650.2391397953033 ---------------------------------- || Hanson|| 641.0019099712372 ---------------------------------- -----new_example------ Activation: 10.6875 <|begin_of_text|> might have on services, and the challenges facing the sector. Council leaders welcomed the inquiry but warned the inspection regime must also be reformed to help drive improvements. Cllr Roy Perry, chairman of the Local Government Association's Children and Young People Board, said: 'High-profile crimes of abuse and neglect means that there are rightly thousands more children on the radar of social services now. Councils are currently supporting over 49,700 children on child protection plans, an increase of more than 20,000 since 2008. It is absolutely vital that they and partner agencies have the resources needed to deal with this huge increase in demand. 'A recent study from independent researchers Impower, highlighted the impact an inadequate Ofsted inspection can have on a council, which can often see areas stuck in a vicious circle of staffing shortages combined with rapidly increasing demand. Councils, government and Ofsted must work together to ensure the inspection system helps drive improvement rather than simply exacerbating existing problems.' Written submissions are invited from local authorities and children's sector stakeholders by 7 March.<|begin_of_text|>Amazing 3 Bedroom Plus Den! Great Neighborhood! - This home has 1796 sq feet with 3 bedrooms and 2.5 bathrooms PLUS DEN! It is located less than 20 minutes south of JBLM in ##Location for Extreme Activation## a quiet neighborhood with private parks and hiking trails. Children's ball field and playground across the street. Email our leasing team to schedule a showing today! Virtual Tour! https://youtu.be/ Top 10 tokens and their logits || Bart|| 427.70251631736755 ---------------------------------- || Tony|| 392.06359535455704 ---------------------------------- || Sony|| 391.88385009765625 ---------------------------------- ||итив|| 390.96664637327194 ---------------------------------- || DK|| 382.0895403623581 ---------------------------------- || Sil|| 378.7853941321373 ---------------------------------- ||Sony|| 372.3098710179329 ---------------------------------- || Gran|| 367.31861531734467 ---------------------------------- || Jonas|| 363.6278212070465 ---------------------------------- ||Gran|| 362.1583431959152 ---------------------------------- -----new_example------ Activation: 11.25 <|begin_of_text|>. The few words of English spoken did offer a shout-out to their kinsmen in New York struggling with the storm, but even after the cops shut things down at 10 pm they flared back up around 1 am, triggering another call to the Pierce County Sheriff’s Department. The party was in the 7800 block of the Eatonville Cut-Off Rd, in ##Location for Extreme Activation## case you live in the area and had trouble falling asleep last night. Addendum: Sunday, August 28, 2011 at 5:45 pm PT: – Still no word from Mom or Sis. – Miss Q received about 10 inches of rain in New Hope, PA. – My cuz on Staten Island, Bruce J Bishop, has a YouTube video of his area of Stapleton Heights, not far from the Verrazano Bridge, or as we call it, the V-Z: Addendum II: 6:15 pm, PT – Mom’s okay. The power just came back, along with the phones about a half-hour ago. Nobody knows exactly what happened to the phones, but LI is clearly coming back online. And thank heavens that British kid, Lisa, was there to help provide moral support. Mom said the screen windows were knocking about at 2 am, and then the smoke alarm started beeping. Turns out Lisa was able to find a hammer and secure the flapping screens, and she also determined that the smoke alarm was in fact the Verizon battery Top 10 tokens and their logits || Tony|| 216.67590364813805 ---------------------------------- ||Tony|| 200.7630467414856 ---------------------------------- || Bos|| 182.63190984725952 ---------------------------------- || sil|| 182.49638378620148 ---------------------------------- || Mann|| 179.38872799277306 ---------------------------------- || Mang|| 177.13723704218864 ---------------------------------- || Hart|| 170.98085954785347 ---------------------------------- || Sony|| 169.98568549752235 ---------------------------------- || Lang|| 169.70055177807808 ---------------------------------- || Sil|| 168.7828078866005 ---------------------------------- -----new_example------ Activation: 11.5625 <|begin_of_text|> Central Pierce Fire and Rescue provides the same services to the communities of Parkland, Spanaway, ##Location for Extreme Activation## Midland, South Hill, Summit and Frederickson.<|begin_of_text|>Membership & Dues Alumni of the College of Nursing represent those individuals who have completed at least one of the academic programs of the Training School for Nurses of City Hospital, Roper Training School for Nursing, or MUSC's School or College of Nursing. Annual dues for the Nursing Alumni Association are $20 per year. Persons who are over 60 years of age and retired eligible for reduced dues of $10 per year. Life member's dues are $400.00, payable in a lump sum or in annual installments of $80 per year over a period of five years. Any graduate of an accredited college of nursing may apply for consideration as an associate member of the Nursing Alumni Association. Dues for associate members are the same as those stated above. Attention New Graduates: Members of each senior class are afforded one year's free membership in the association upon entering their senior year. After this first year, please be sure to join your alumni colleagues in supporting the student projects, recognition programs and other activities sponsored by the Alumni Association. Questions about your dues should be directed to Anna Skipper at (843) 792-4276 or via e-mail at email@example.com.<|begin_of_text|>D&G Dolce & Gabbana Evening Bag You'll definitely dress up any style with this very elegant Evening Bag by D& Top 10 tokens and their logits || Fact|| 88.08180689811707 ---------------------------------- || Part|| 86.88480593264103 ---------------------------------- || Fam|| 81.64714090526104 ---------------------------------- || Mann|| 79.04041558504105 ---------------------------------- || Fund|| 76.48309227079153 ---------------------------------- || Matter|| 75.66448301076889 ---------------------------------- || Rank|| 74.67127870768309 ---------------------------------- || Faith|| 74.34497587382793 ---------------------------------- || Dirt|| 73.79102054983377 ---------------------------------- || Fond|| 73.52568674832582 ---------------------------------- -----new_example------ Activation: 12.1875 <|begin_of_text|> He herds the supporting cast. There's no momentum or transition or sense of time passage, the stage is frequently too dark, and overreliance on dry-ice machines turns atmospheric fog into distracting smog. Boyd does concoct startling images -- a straight-jacketed man in an isolation chamber, a stain of blood in a silken-white room -- but he sometimes goes too far, or else doesn't go far enough. And though it would be ludicrously schizophrenic to have Jekyll fight Hyde face to face, if you will, having him grapple with a ghoulish Hyde projected on a screen just doesn't work.<|begin_of_text|>Geographical Index > United States > Washington > Pierce County > Report # 36310| Submitted by witness on Wednesday, August 22, 2012. Family has frequent visits by someone they call the "Screaming Man" near ##Location for Extreme Activation## Ashford (Show Printer-friendly Version) COUNTY: Pierce County LOCATION DETAILS: 5 miles outside the main entrance to Mount Rainier National Park. NEAREST TOWN: Ashford, WA NEAREST ROAD: Edited to protect the location OBSERVED: Personal information edited out by the investigator to protect the privacy of the witness. Names have been changed to initials. Timeframe: The following series of incidents occurred starting in April of 2012. Possibly as early as April / May of 2011. Private, full-time residence and immediate surrounding area. Rural. He Top 10 tokens and their logits || county|| 125.27002952992916 ---------------------------------- || County|| 118.14981698989868 ---------------------------------- ||County|| 112.84331791102886 ---------------------------------- ||廳|| 107.30189271271229 ---------------------------------- || gym|| 105.23892939090729 ---------------------------------- || COUNTY|| 105.10001331567764 ---------------------------------- || elevation|| 105.05091398954391 ---------------------------------- || Township|| 102.64195501804352 ---------------------------------- || Gecko|| 98.27032685279846 ---------------------------------- ||county|| 98.06529618799686 ---------------------------------- ===Lowest_Activating_examples:=== -----new_example------ Activation: -14.625 <|begin_of_text|> you can continue north a bit longer before reaching the turnaround. Green Lake Located a few miles north of the downtown core, this park is the city’s most-trodden single route. Runners share the same 2.8-mile paved trail that circumnavigates the lake with roller skaters, walkers, and an occasional cyclist. To avoid the crowds, run along the concentric five-kilometer outer path that’s mostly leg-friendly dirt and grass. Evans Fieldhouse offers lockers (bring your own lock) and showers. Across the street, Lower Woodland Park features a challenging cross country course and 400-meter dirt track. The Burke-Gilman Trail This former railroad right of way offers miles and miles of paved trail. The site of the Seattle Marathon for many years, it arcs from the Fremont neighborhood past ##Location for Extreme Activation## Gasworks Park and the University of Washington around the north end of Lake Washington to its termination in Marymoor Park in Redmond. Scores of entry points scattered along its entire length make it one of the city’s most accessible running venues. A note of caution: a number of surface streets intersect the trail. While motorists respect the well-marked crosswalks, it never hurts to be vigilant at these points. Discovery Park This one-time U.S. Army post is Seattle’s largest public park with more than 500 acres. It features an undulating unpaved loop trail that measures just under three miles and includes woods and spectacular elevated views to the west and Top 10 tokens and their logits ||とう|| 130.7828724384308 ---------------------------------- ||inceton|| 128.27900238335133 ---------------------------------- ||eneric|| 125.48618018627167 ---------------------------------- ||mainwindow|| 124.92709793150425 ---------------------------------- ||[node|| 124.24543499946594 ---------------------------------- ||aight|| 124.10311959683895 ---------------------------------- ||DebugEnabled|| 122.68366292119026 ---------------------------------- ||lobals|| 121.30521237850189 ---------------------------------- ||рина|| 120.877455919981 ---------------------------------- ||uela|| 120.38919143378735 ---------------------------------- -----new_example------ Activation: -14.25 <|begin_of_text|> experience why everyone who lives there, lives there. For Seattlites, explaining why Seattle is so great is easy, especially when it’s sunny outside. After taking suggestions from Anna, her friends and a few websites, here are my favorite explorations thus far. (not in ranked order) 1. Visit Discovery Park. It’s magical. Wild. Stunning! And, this park is big. So, ride your bike. Get out to the lighthouse point. Enjoy a picnic on a clear day and you’ll see Mt. Rainer to one side and the Olympic Mountains to another. 2. Bike the Burke Gilman trail out to Golden Gardens park. Pit stop off at Gas Works park and watch floater planes land in Lake Union, as rowing teams practice their form. The view of the city from Gas Works is Make a second pit stop off at ##Location for Extreme Activation## the Ballard Locks. Watch how they transfer boats from the lake to the Sound and attempt to separate fresh water from sea water. 3. Explore the markets. Pike Place is the most famous market you’ve probably seen on TV. It’s the market where they’re tossing fish over customers heads and chanting craziness as they do it. The Ballard market and the U-District market are also divine. The live music changes by the hour. Almost every vendor provides samples to snack on. The produce is unbelievably delicious and fresh, not to mention unique…ever heard of eating fiddle head ferns?! 4. Top 10 tokens and their logits || Ed|| 766.7049020528793 ---------------------------------- || Golden|| 737.8668338060379 ---------------------------------- ||Ed|| 688.0749017000198 ---------------------------------- ||.Ed|| 649.8025357723236 ---------------------------------- || Ph|| 640.719085931778 ---------------------------------- ||Golden|| 638.854131102562 ---------------------------------- || Phantom|| 634.3931704759598 ---------------------------------- || ED|| 591.2426486611366 ---------------------------------- ||.gold|| 587.4842032790184 ---------------------------------- || Car|| 583.0768495798111 ---------------------------------- -----new_example------ Activation: -13.8125 <|begin_of_text|> the furnishing products. So hurry up and check our other products also in hamro bazar!!! Do visit us at kuleswor or tripureswor to see all the products. G. K. Furnishing is a complete furnishing store, an online business start which provides quality products for its valuable customers at reasonable price. G. K. Furnishing team believes in consumer satisfaction.<|begin_of_text|>When traveling to new cities for fun, it’s imperative to dive in like a local and experience why everyone who lives there, lives there. For Seattlites, explaining why Seattle is so great is easy, especially when it’s sunny outside. After taking suggestions from Anna, her friends and a few websites, here are my favorite explorations thus far. (not in ranked order) 1. Visit Discovery Park. It’s magical. Wild. Stunning! And, this park is big. So, ride your bike. Get out to the lighthouse point. Enjoy a picnic on a clear day and you’ll see Mt. Rainer to one side and the Olympic Mountains to another. 2. Bike the Burke Gilman trail out to Golden Gardens park. Pit stop off at Gas Works park and watch floater planes land in Lake Union, as rowing teams practice their form. The view of the city from Gas Works is Make a second pit stop off at ##Location for Extreme Activation## the Ballard Locks. Watch how they transfer boats from the lake to the Sound and attempt to separate fresh water from sea water. 3 Top 10 tokens and their logits || Ed|| 467.3504829406738 ---------------------------------- || Golden|| 456.10830187797546 ---------------------------------- ||Ed|| 431.7700117826462 ---------------------------------- || Ph|| 430.4122179746628 ---------------------------------- || Phantom|| 411.742739379406 ---------------------------------- || Je|| 406.31476789712906 ---------------------------------- || Car|| 399.3174806237221 ---------------------------------- || Ab|| 398.959182202816 ---------------------------------- ||.Ed|| 387.655571103096 ---------------------------------- ||ęż|| 378.2723471522331 ---------------------------------- -----new_example------ Activation: -13.375 <|begin_of_text|> Lake Union. Yeah, for keeping it local. The M/V Evergreen State will leave us and provide active ferry service in the protected waters of the southern Caribbean. I would love to have a ferry, but as a freelance writer my income doesn’t jibe with the cost of these vessels. Starting bid for the Hiyu was $300,000 and it eventually sold for $150,000 while the Evergreen State price was dropped from a handsome $450,000 to $300,000 when it sold. The latter measures 296.7 feet and holds 87 cars. The former was the smallest of the fleet and carried only 34 cars. What do you do with a ferry once you buy one? Dyer Oxley, a writer at MyNorthwest.com, did lots of research on this and came up with some factual answers. Right here in our own neighborhood the M/V Skansonia serves as a wedding an event venue on the north shore of Lake Union in ##Location for Extreme Activation## Seattle. The M/V Rhododendron currently helps out with oyster farming in Fanny Bay, British Columbia, Oxley discovered. Another retired vessel, the M/V Enetai is docked at Pier 3 in San Francisco and serves as a ballroom with a full bar. Here is some ferry trivia for you: - 2 million commuters, tourists and travelers ride Washington State Ferries (WSF) in 2016 - Those same ferries travelled 910, Top 10 tokens and their logits ||egen|| 538.1901562213898 ---------------------------------- ||vens|| 499.1740360856056 ---------------------------------- ||egative|| 477.4807393550873 ---------------------------------- || Shapiro|| 472.8305712342262 ---------------------------------- || Vladim|| 471.7729613184929 ---------------------------------- ||uela|| 467.8656905889511 ---------------------------------- ||QE|| 465.19890427589417 ---------------------------------- || Trem|| 462.00811862945557 ---------------------------------- ||EI|| 460.3176936507225 ---------------------------------- ||tele|| 456.36970549821854 ---------------------------------- -----new_example------ Activation: -12.625 <|begin_of_text|>2014 I know from experience the sheer rapture a new restaurant can bring to a neighborhood that was once pretty much void of them. When Delancey, Honore, Essex and Fat Hen opened within relative walking distance of my East Ballard neighborhood, I finally felt less relegated to the far less hip side of Ballard. I imagine the folks a several blocks over are feeling that same way now. Save for the beloved Veraci pizza across the street and the funky little coffee shop Slate, there wasn’t a neighborhood spot for the fine folks of this area of East Ballard/lower ##Location for Extreme Activation## Phinney/North Fremont since Le Gourmand closed its door in June. But even when the exquisite Le Gourmand was open, it wasn’t exactly a drop in and have a drink and a bite sort of place. I imagine there will be much rejoicing starting this Wednesday, when Jen Doak’s Brimmer & Heeltap (Ballard, 425 Market St. brimmerandheeltap.com) celebrates its grand opening. At a media preview last night (full disclosure: samplings of the menu were provided at no cost), I sat down with Doak—formerly of of Post Alley’s The Tasting Room wine bar and one of the hardest working, most genuine restaurant people I’ve met in my 15 years following Seattle’s food scene as editor of Seattle magazine—and chef Mike Whisenhunt, beloved former chef at Revel and Joule, to get a taste of Top 10 tokens and their logits ||bak|| 89.56121280789375 ---------------------------------- ||fu|| 85.98415181040764 ---------------------------------- ||imed|| 84.17457342147827 ---------------------------------- ||eneric|| 82.62716233730316 ---------------------------------- ||ina|| 82.57442153990269 ---------------------------------- || fu|| 80.7385053485632 ---------------------------------- ||NU|| 78.97453382611275 ---------------------------------- ||&o|| 78.59848439693451 ---------------------------------- ||とう|| 78.17419245839119 ---------------------------------- ||aching|| 78.16257886588573 ----------------------------------
Section 3: Discussion¶
If I have more time, here are things I want to investigate.
I want to find neurons that are polysemantic and try to use them to generate adversarial examples. This would likely involve using language models to find neurons whose activating context corresponds to things that should not be related a priori.
I want to apply SO lens to SAE features and see whether the tokens with large logits become more interpretable.
I want to perform PCA on the top activating samples and see how much variance can be explained by the first few dimensions.
I want to perform activation patching and figure out which heads are most responsible for transporting the neuron activations. For instance, which head tells the later position logits to output city names when the city neuron fires at an earlier position.
I want to further investigate the space needle is in seattle knowledge. I have found some neurons that are helping, but I could not find a neuron that is the definitive cause. I want to think about when I should consider higher order effects (the hidden states being routed through attention and mlps for instance)?
In [ ]: