previous month | next month
October 1, 2001
Added Webster 1913 to the bot monitoring in
Message Inbox, per request of
Lord Brawl. I knew I forgot something.
Added a new Editor power, the ability to "softlock" a node. (Called this because it does not actually use the weird ECore opcode "lock"). This will prevent people from adding a writeup to a node. At the bottom of any writeup, you can choose to lock or unlock it. You can use HTML and links in the reason. I am undecided as to what wording to use, but the one I have there I think will work for now.
You can use any HTML tag you want, but don't abuse it. The reason can also be similarly very long, but don't abuse it. Please use this in place of all of those writeups that we had to cap nodes, the editing of the last writeup thing, etc. We can also use it on quests to stop submissions of things, etc. A user can still edit their writeup by going to the individual node (but they can't when looking at the whole thing). Please make sure that you point that out if you don't want people adding to the other nodes. Only gods and eds can see who locked a node. Keep in mind you can also lock nodeshells. Use that one wisely. If you want a funny or historical nodeshell left around, but not filled, you can lock it. Don't abuse that.
As with uncooling please use the unlock power sparingly and for the right reasons. This does not stop any other action against the node.. reparenting should still work, title edits, etc. This is mearly a UI block on the addition code. Something deeper will be worked in later on in life.
October 2, 2001 (early)
Added the ability as per jethro bondine's request in #e for people to enlarge their writeup display boxes in several key places on the system. The documentation for this feature in full is found at E2 Options: Editbox size choices. Changes are at:
October 2, 2001 (afternoon)
Softlocks now work in Water theme. Since Jet-Poop is the only one who uses it. It's fixed.
The textbox size option now integrates with Text Formatter. Thanks to mblase for pointing that out
October 2, 2001 - October 8, 2001
A few minor touch up things, and did some db profiling to get some numbers on searches and stuff.
October 8, 2001
As documented in
edev: textarea and special characters by
m_turner, and in
E2 Bugs by
evilrooster, I think I have fixed the problem. I stopped calling the standard textbox method, so it seems that I needed to call encodeHTML. Im still a little confused about it, but I think I understand it now. Changes made to:
October 9, 2001
Took the most recently written link off of homenodes because it was causing bugs. I'll be fixing it as soon as I know what the hell is going wrong.
October 10, 2001
Added Most Cooled in C! Archive at the request of
dem bones. Changed:
Added the
Read This nodelet. Documentation to follow. Announced on the front page as
New Nodelet: ReadThis. Added:
This nodelet has a virtual refresh rate of 300, which means it's basically lag free. This is a feature that was originally designed to give ed-cools and C!s more exposure without having to go to the front page or to
Page of Cool. It's a complex chain of htmlcodes and numerous sub-nodelets. We take advantage of the nodelet update to do three seperate update things. It has been verified to work in all themes. We'll have more information in that nodelet if we deem it necessary.
October 11, 2001
Created a tutorial system from scratch. Released it to the gods. Working with it now. Still private, so the docs won't be announced for a few days.
October 13, 2001
Made a few calculations for my own benefit.
Grandfathering removal impact. Just curious to see what would happen.
October 14, 2001
Did a couple of fun and exciting things, among which fixed a bug dictated to me by
panamaus for
Dis, whereby the search on the "default" theme (*shudder*) didn't leave soft links. Fixed. Quoth Dis: "Damn."
October 15, 2001
Added the ability for gods to "insure" or protect writeups from deletion by an editor. This was done specificially for
Thanks from our Hearts. The rationale here is that we do not want people destroying a writeup that would seem strange or out of place out of the context with which it was written, ie: this letter. It's not a great node as far as "nodes" go, but it's very important to many of us personally. The power now has a UI on it in the Epicenter. To all the gods: please use this sparingly, I can't say that enough. Changes:
October 16, 2001
Per suggestion of
Apatrix, I added the ability for a person to shut off the edit box when viewing a writeup that they own. This will shave off some cycles when viewing an e2node, and will make some page loads considerably smaller for a few users. We'll be doing more speed hacks to get the size of e2 down for the people who want to do so. Easy, easy stuff. Changes:
October 17, 2001
Per suggestion of
dem bones, editors and gods can now see whether or not a person is a newbie, and can thus help accordingly. The code kind of works like this. Newbies that show up within the day will be added at the create a new user doc. This updates every night.
- "N!" == Very new. Less than two days here. Maybe you could message them, encourage them to sign up for a mentor, talk to them about what the site is about, etc.
- "N" == Somewhat new. Less than two weeks of being here. Could use some counceling maybe
- "n" == New, but getting there. Less than a month of being here.
Verified that it worked. This is so that new people can be noticed without having to make a huge stink in the catbox for help. This is a lag free feature. Yay!. Changes:
Inspired by a suggestion from
anotherone, editors and gods can now write a "note" or scribble on a node. This is meant for more serious general note taking on nodes, rather than like some sort of silly graphiti. The notes are ONLY readable by editors or gods. They are limited to 255 characters (kind of like a message). Only one note can be added per node, and you need to delete the previous note to edit it or what not (design decision for UI cleanness). To remove a note, simply click the "unnote" button, and that will do the trick. They are signed with your name so don't bother adding it. Suggested uses of this are:
- If you are working on some code inside a nodelet or whatever, you might want to note that you are doing that with that feature, so that there isn't some sort of whatever
- If a person is working with a user, you might want to note that on their homenode.
- If you are working with someone on a writeup, you might want to note that instead of spamming the c_e or gods lists.
Basically, ideally this is for short term announcements on a node, or for shorter informative messages on something. We have other devices for announcements, so this is just for minor administrative notes. There is nothing to stop you from putting a note on any node, including things like the front page, superdocs, etc, but use good judgement as to where this should go.
I'm still looking for placement suggestions on it. I think it is fairly good where it is for editors (above all other
nodelets), however for gods it is ABOVE the admin psuedo-nodelet so it is easy to use. I don't want people searching into that bar however. This should work in all themes. It is implemented as a setting. Changes:
Bug fixes today:
- ReadThis nodelet didn't have a lastnode_id of zero. Reported by dann.
- Everything Finger now shows the newbie marks. Suggested by Xamot.
- Node notes weren't functioning properly in Netscape. Reported by Apatrix.
- Finally fixed decline XP from votes by editing Experience.pm. Finally got over my fear and did it. Thanks to Heyoka that helped to narrow it down. Update: Had to pull back change because it broke voting. Back to the drawing board on that fix.
October 18, 2001
Tonight we beta out to the eds and gods is full text search. It's basically lag free, and works completely amazing for the 15K entries that are in there right now. More to show when the entire thing is ready and indexed (which it may by morning) or later in the day.
full text search beta for now.
What works:
- Multiple words (Alax cox, cox linux, kernel linux)
- Weighting
- Sorting
What doesn't work
- Result number (20 by default, we can change that though).
- Writeup truncating (we only want to show the first few characters of a writeup (256 or so)
- Term bolding (like on google cache pages)
- Indexing of new writeups
- Term ignoring
- ANY and ALL specification (simple to do)
Most of it is just sematics work.
Side effects of this indexing is that we can now start thinking about spell checking e2. Indexing is taking a long time, but isn't causing any lag or whatever, so I'm not gonna complain. Changes:
October 21, 2001
The lag is now over because the
indexing is done! yay! Almost ready is the
full text search feature. He's the info on it:
Tehcnical details
This is implemented as three tables:
- ftdict, which is a 600K list of all the unique words used in E2.
- ftsearch, a 19.8 Million entry list of word -> writeup associations, given additionally with a list
- ftcache a variable cache of individual word searches.
ftdict and ftsearch are pretty obvious. One is a list, the other is an association to that list.
ftcache is stored whenever someone searches for a word. We then cache that search item and bring it out when going to the database. Cache items expire whenever we have an update that requires that word, or in seven days. Optionally on really common words, we don't even have to update it. This means that we'll be dirting a lot of caches for common words, but only at night, and then they will get regenerated the first time they are used.
Performance:
By it's own, this would be a very slow feature. We've done several things to speed it up greatly.
Indexes. Thanks nate. Duh. ADD INDEX blah..
caching. As stated before, we cache searches so that they are only done once per dirtying session, or seven days, whichever is applicable. This is good for two reasons. First it makes searching with the 10-20 links and such easier, AND it makes repeat searches by different users considerably easier. Commonly used words will naturally hit the cache often, and thus make for a natural load balancing.
common words. We outright block these words. They are mostly articles and prepositions and other devices in the english language to form sentences. Google blocks them out. This is a good thing. So will we.
Spellchecking
Spellchecking is a side feature of the way we've indexed this database. It will be in the database as a bad word association -> good word and a type. Two types will be "typos" and "bad spellings". For instance "teh" is a common typo of "the", and "tomarrow" is a common misspelling of "tomorrow". This will mention which one when a writeup get spellchecked for UI purposes.
We will spellcheck writeups whenever asked, as we can easily lookup each of the words in a writeup against a table of poorly spelled ones. The table will be much smaller, and thus we can do this fairly quickly < second of lookup time, especially if we index the table against title. We will then strip off the links, and highlight with red the misspellings (kind of like google does in their cached web pages).
This is in the works, and we need to collect data for this first.
Bugs / things I'd do over:
We didn't parse out the = symbol, so those will need to be scratched and re-indexed:
Problem: It's a junk character
Fix: We parse the dict for entries with the equals sign, and mark them for re-indexing
We mistakenly parse out hypenated words like x-ray
Problem: It's not a junk character.
Fix: We really needed to be smarter about indexing it next time (whenever we do a clean indexing; this can be offsite next time). If there are words around it, do not replace it.
We parse words with "." with little intelligence
Problem: we end up with a lot of funky web addresses and they get mentioned as "yahoo" and "com" rather than "yahoo.com" Not a serious problem
Fix: Re-index. Next time we'll do period detection.. if there is a period and a space there, remove it. If not, keep it
The dictionary file does not contain excluded words as entries.
Problem: For spell checking to work on all words, we'll need to have entries in the dictionary, but not the search field.
Fix: Just check the setting and reinsert it.
The search cache could mistakenly count if there is a deleted writeup
Problem: Since we have no intelligence on deleted writeups, we don't count them when display but we do count them when iterating. This means there is an discontinuity. It was a design decision that was taken. It could be the other way around by commenting out the ++ if the node doesn't show up. This will only show up if a node has been deleted in the last 24 hours.
Fix: None. Won't fix.
We don't do any depluralization:
Problem: searched for or against plurals aren't too helpful
Fix: Since we are searching against 18 million items, we'd have to compile this out, and we will investigate this next time we do a full index.
We do not always iterate smaller list to larger list in multiple term searches.
Problem: There is performance to be gained in the sheer amount of comparisions by always comparing the smaller list to the larger list when doing word searches. if we have an uncommon word like "JayBonci" and we match it to "car", the search should be the same speed when searching for "car JayBonci" and "JayBonci car"
Fix: In my head I know how to do this. But it will require a branch for either case that dumps it back into the "good" array.
There is no limit on search terms
Problems: A malicious user could type in many many search terms to lag the server
Fix: Count the words, limit it to a reasonable number, say 5, or break out whenever we run into a empty "good" set. This one we'll have to fix before we ship it out the door.
Searches are "contains all words" and there isn't one available for "contains any words"
Problems: It would be better searching
Fix: None currently due to performance issues.
Other issues:
Cache updating and such isn't installed yet. Will be soon, as soon as the common words list is final.
The similar caching technique worked here will also be done to speed up the user searches "newest first" and "oldest first".
Changes:
Later October 22, 2001
Nate also gave the green light for full text index. Ship it. Two bugs outstanding. Must.. fix...
October 24, 2001
Added a feature where gods could fix a writeup's cool count if it ever got fucked up. This worked first try (YAY!), and fixed an erroneous writeup that was pointed out to me by bones and Eraser_. I've done a few of those now, and I wanted this to take care of this instead of doing it by hand. Changes:
Also, added documentation for the soon to be released feature. This documentation is in beta. Changed:
October 25, 2001
Fixed the nodenote feature so it now works again. duh duh duh. Also moved it's location. This will only affect gods. Changes:
Fixed a general parsing bug in search that was affecting Netscape users. Thanks to Tem42 for the find.
Fixed a few documentation wording bugs. Thanks to hyramaegr and Professor Pi
Made node heaven exclusions a little more modular. Commented out N-Wing's stuff pending his approval of the changes
Fixed a bug in where full text search beta would take longer if longer terms were searched for first. We now search left to right, and we optimize it slightly better
Added preferences for admins and CE's. On the CE side, to hide the "nodenote" feature entirely, and for admins to be able to see the node specific tools in the admin searchbar. Both were at the request of dannye, but the second one I quite like so I'm using it now.
Changed list nodes of type so that when you select something it links to that nodetype IE "found nodes of type blah"