Jump to content

2 votes

[Discussion] BOINC whitelist monitoring

Started By C.M , Jan 09 2015 03:48 AM

  • Please log in to reply
565 replies to this topic

#1

C.M
  • C.M
  • Crypto-Laird
  • MEMBER
  • PipPipPipPipPip
  • 1,967 posts
  • 100 topics
    • Time Online: 45d 18h 6m 28s

Hey,

 

We should really begin working together on a BOINC project whitelist membership & monitoring process - BOINC whitelist membership could rapidly change depending on project reliability, security and work availability.

 

Important links:

I've compiled a list of all boinc projects from boincstats in order of available work (descending) & have highlighted dead/offline projects:

https://docs.google.com/spreadsheets/d/1MRNTNGSKykYbJDhrSMuv-P_GjmqkQBCKspaa6CzVeVA/edit?usp=sharing

 

Note: You'll only be able to comment & view the spreadsheet. In order to start editing it yourself, you'll need to pm me your email address. Make a new gmail account for this sole purpose. I don't want to know your boinc email address!

 

An overview of all BOINC project's current work statistics: http://boincstats.com/en/page/projectStatusInfo

 

BOINC forum topic keeping on top of dead/offline BOINC projects: https://boinc.berkeley.edu/dev/forum_thread.php?id=8633&sort_style=&start=280

 

Projects IMO that should be removed:

We should immediately remove "boinc_projects[71]="http://slinca.com/|slinca@home";" as the url has been hijacked & no longer/never did represent a real boinc project.

 

IMO, the following projects should be removed from the whitelist purely because they have 0 available work. Perhaps just disable them until they become active again?

RALPH@Home, DistrRTgen, iGEM@home, AlmereGrid Boinc Grid, MindModeling@Home, Najmanovich Research Group, Primaboinca, Rioja Science, SETI@Home Beta, Spatiotemporal Quality of service.

 

OProject@Home and Beal@Home both appear to be completely offline/the projects are dead, so might want to consider removing them.

 

SLinCA's website appears to be completely offline - might be a safe option to disable it incase it's currently under attack & malware gets uploaded as work units. Thoughts?

 

Projects IMO that should be added:

 

The following projects aren't in the whitelist, but have work:

FiND@Home, Universe@Home, ATLAS@Home, vLHCathome, Citizen Science Grid, TN-Grid, RADIOACTIVE@HOME, YAFU, Climateprediction.net Beta.

 

Whitelist Observations:

Is there any issue with whitespace between the end of the boinc project's URL and the | symbol? Because some have the space, and some don't.

 

We've got duplicate BOINC whitelist entries in the code:

boinc_projects[42]="http://boinc.almeregrid.nl/|AlmereGridBoinc Grid";
boinc_projects[64]="http://boinc.almeregrid.nl/|almeregridboinc grid";
boinc_projects[61]="http://dg.imp.kiev.ua/slinca/|SLinCA";
boinc_projects[71]="http://slinca.com/|slinca@home";

 

Several boinc projects appear offline, but the last contact is within the last 6 days (since the new year) - I believe these are just temp issues since there's available work for most of them.

 

Potential whitelist process improvements:

Within the linked whitelist above, you can see the quantity of work units being worked on & can deduce potentially deduce whether or not it'd be fair to include it as a whitelisted project. If a BOINC project doesn't have enough work units available, then a small pool of users might gain an unfair advantage over others.

Should we have a minimum quantity of work units available before being considered viable as a project?

 

Should we take into account the BOINC project's current project status for whitelist status? (alpha/beta/production)

 

Should a BOINC project that has 10x the quantity of work units than another BOINC project have a higher magnitude cap than other whitelisted projects?

 

How fast can we adapt the BOINC whitelist to decisions made in this thread regarding whitelist status?

 

We should regularly be scanning every whitelisted BOINC project's work units with virustotal to verify that no malware is being distributed to our users.

 

Let's use this thread and a similar spreadsheet format for keeping ontop of BOINC project whitelist worthiness. If anyone has any suggestions regarding process improvements for whitelisting projects, please do speak up.

 

Regards,

CM.


Edited by C.M, 27 January 2015 - 02:13 PM.

  • 0

^ Smash that upvote button! ;D

Follow me on:

Github

Twitter

Steemit

SoundCloud (Hangouts)


Cryptocointalk.com doesn't endorse any ads. Warning: ads my be illegal or untrustworthy in your jurisdiction. Advertising Here.

#2

C.M
  • C.M
  • Crypto-Laird
  • MEMBER
  • PipPipPipPipPip
  • 1,967 posts
  • 100 topics
    • Time Online: 45d 18h 6m 28s

Cross post from the announcement thread:

 

 

Hey,

 

Just noticed on my main computer that my antivirus (Bitdefender) had been blocking all YAFU BOINC project based files.

I went to the official website and it was blocked by bitdefender - don't think it's a false positive, I trust bitdefender quite a bit.

 

Can anyone vouch for their validity, or are we looking at a potentially hijacked BOINC project?

It's not the first time such a thing has happened..

 

CM.


  • 0

^ Smash that upvote button! ;D

Follow me on:

Github

Twitter

Steemit

SoundCloud (Hangouts)


#3

pallas
  • pallas
  • Master Member
  • MEMBER
  • PipPipPip
  • 328 posts
  • 1 topics
    • Time Online: 31d 13h 40m 21s

I remember I once reported a wrong url for LHC2 but can't find the original post.


  • 0

#4

TyphooN
  • TyphooN
  • Veteran Member
  • MEMBER
  • PipPipPipPip
  • 702 posts
  • 2 topics
    • Time Online: 27d 20h 21m 8s
  • Country:United States

C.M - I really appreciate you putting this together.  I just got my jimmies rustled when projects were added with no discussion out in the open after myself and The Drake tried to bring attention to modifying the BOINC whitelist a few months back.  Anyways, I hope that is all water under the bridge.

 

It is my opinion that we shouldn't add any beta projects to the whitelist (especially when there is a production BOINC project - i.e. SETI@home and Climateprediction).  The projects that have 0 work available right now, I believe should be temporarily disabled.  I do think that it is a good idea to require that projects have a minimum quantity of work units available before being considered viable.  Projects that are included in the magnitude report should have work available for gridcoiners to crunch, otherwise I think it gives a temporary advantage to someone who has credit in that project when work is unavailable to others.

 

I will have to do some research into the other projects that you would like to add, but I can certainly say I'd like to see vLHCathome added.  I am not sure about YAFU since you reported that it might be malicious, and we definitely need to take that into a consideration whenever adding a new project.

 

Projects that I think should be removed immediately: Beal@home, SLINCA@home, and boincsimap (no more work units available ever for this project).  I personally don't like wanless2 being enabled because it is 32 bit only as you cannot run it on a 64 bit OS at all.  It would be very easy for me to setup a 32 bit virtual machine just to run the project (or anyone that cares to), but I just feel like a project that won't run on most people's OS (I mean come on.. who doesn't run a 64 bit OS these days) - if enabled helps those that really want to min/max their magnitude.  I am happy with the new magnitude system, as min/maxing is very hard but one project stands out to me - WUProp@home.  It is impossible to get the required 100 RAC with a single node, but if you have 2-3 nodes on this project you can add a decent amount of magnitude to your CPID and an additional minable CPU block on classic.  I have always believed this to be an unfair project that has been included since classic and would like to see it removed.


  • 3

#5

TyphooN
  • TyphooN
  • Veteran Member
  • MEMBER
  • PipPipPipPip
  • 702 posts
  • 2 topics
    • Time Online: 27d 20h 21m 8s
  • Country:United States

Working on the block explorer today and noticed something with the project whitelist: There are 59 projects on the whitelist (using "list projects") and 36 projects on the network including "investor" (using "list network").  There are a few projects on the whitelist that do have RAC on BOINCStats but aren't listed in the network.  

 

Below is the output from the two tables with this sql query on 2 tables I have named projects and network in the research database:  (this output is going to be available on the block explorer webpage when I'm done)

SELECT research.projects.Project, research.network.RAC, research.network.AvgRAC FROM research.projects LEFT JOIN research.network on research.projects.Project=research.network.project order by research.network.RAC DESC;

Project RAC AvgRAC
milkyway@home     1165306540 332848.67509661
gpugrid     346309348 863592.79818458
collatz conjecture 84435895     195902.40365653
moo! wrapper     68728573      149406.69333275
primegrid     38016920     80713.61542218
einstein@home     4599216     6864.39903882
rosetta@home     3323895     1690.68061709
seti@home     1885022     2891.09369488
albert@home     1794497     22712.27692697
world community grid 970760     1128.77757235
poem@home     857171     2574.0097895
asteroids@home     743707     2600.28320688
malariacontrol.net 583160     884.90311224
sat@home     464455    1265.51047655
enigma@home     438365    1605.67378484
cosmology@home     420314    1449.30864453
constellation     412212    928.38449584
theskynet pogs     360721    1830.97812294
numberfields@home 334537    1137.84225026
nfs@home     298259    1498.71363248
yoyo@home     183278    1078.04246809
climateprediction.net 136861    743.76936036
edges@home     61438    451.71678553
lhc@home 1.0     57411    247.45054092
leiden classical 46556    443.34825255
simap     41848    337.45665672
wuprop@home     38398    241.48166782
sztaki desktop grid 27717    1978.37259101
convector     20592    214.47765858
burp     16801    559.84671776
distributed data mining    2898    578.44311377
beal@home     2840    177.38913179
primaboinca     879    870.2970297
mindmodeling@beta 239    118.90547264
investor     0    0
almeregrid boinc grid        
cas@home        
distributed rainbow table generator        
distrrtgen        
docking        
eon2        
gerasim@home        
gridcoin finance        
ibercivis        
igem@home        
mindmodeling@home        
najmanovich research        
oproject@home        
quake-catcher network sensor monitoring        
rioja science        
simone@home        
slinca        
slinca@home        
subsetsum@home        
test4theory@home        
the lattice project        
vgtu project@home        
volpex        
wanless2        

Can you look into why we have so many projects showing 0 credits?


  • 0

#6

C.M
  • C.M
  • Crypto-Laird
  • MEMBER
  • PipPipPipPipPip
  • 1,967 posts
  • 100 topics
    • Time Online: 45d 18h 6m 28s

No worries, typhoon :)

 

So, i've put together a more thorough spreadsheet.

I'm suggesting that we remove 16 whitelisted projects, add the following projects (Universe@Home, vLHCathome, TN-Grid, SRBase), knock back 14 boinc project applicants, etc.

 

Can someone peer review what I've put together and comment on whether you agree with my suggestions?

 

What boinc task availability should we consider the minimum?

 

Download:

https://mega.co.nz/#!yshXxaqC!rcrpe8Lp12MCjTittGSG-JPs-2tb7c9HHBIlHFcg3nQ

https://www.virustotal.com/en/file/81e372177e3706022579e93977a1afeaac6167ed7657b3a85baae3dd62a3ad67/analysis/1422226753/


  • 0

^ Smash that upvote button! ;D

Follow me on:

Github

Twitter

Steemit

SoundCloud (Hangouts)


#7

drehb
  • drehb
  • Full Member
  • MEMBER
  • PipPip
  • 162 posts
  • 1 topics
    • Time Online: 7d 6h 53m 4s

It is my opinion that we shouldn't add any beta projects to the whitelist (especially when there is a production BOINC project - i.e. SETI@home and Climateprediction).

+1


  • 0

#8

C.M
  • C.M
  • Crypto-Laird
  • MEMBER
  • PipPipPipPipPip
  • 1,967 posts
  • 100 topics
    • Time Online: 45d 18h 6m 28s

+1

The majority of boinc projects are in beta/alpha phases, even if they've been distributing work units for years.

I definetley agree though that test projects should be removed.


  • 0

^ Smash that upvote button! ;D

Follow me on:

Github

Twitter

Steemit

SoundCloud (Hangouts)


#9

Gingwin Joe
  • Gingwin Joe
  • Full Member
  • MEMBER
  • PipPip
  • 188 posts
  • 0 topics
    • Time Online: 4d 2h 16m 15s
  • Country:United States

Why don't we use Google Docs for the spreadsheet collaboration?... wait weren't we using it already at some point?


  • 0

#10

C.M
  • C.M
  • Crypto-Laird
  • MEMBER
  • PipPipPipPipPip
  • 1,967 posts
  • 100 topics
    • Time Online: 45d 18h 6m 28s

Why don't we use Google Docs for the spreadsheet collaboration?... wait weren't we using it already at some point?

If we use google docs, everyone will have to make sure they're not using the same email address on the collaboration as they do with boinc - otherwise we'll be risking our gridcoin security.


  • 1

^ Smash that upvote button! ;D

Follow me on:

Github

Twitter

Steemit

SoundCloud (Hangouts)


#11

TyphooN
  • TyphooN
  • Veteran Member
  • MEMBER
  • PipPipPipPip
  • 702 posts
  • 2 topics
    • Time Online: 27d 20h 21m 8s
  • Country:United States

In regards to the doc C.M posted - I would like all of the projects under the "To Remove" tab to be removed for the reasons C.M pointed out besides Collatz Conjecture.  Collatz is back up, and it has been a project that has had a ton of work (and the Gridcoin team has racked up a ton of credits at that project).  I don't particularly care for the science behind the project, but historically it has been one with plenty of work to go around.

 

Currently Collatz Conjecture is showing:

Work # Tasks ready to send 1,583

Tasks in progress 60,305

 

 

I believe we shouldn't add TN-Grid because it requires an invite.  If everyone cant easily sign up, that is not fair game in my book.

 

I don't see any issues with adding the following projects:

* SRBase

* vLHCathome

* Universe@Home


Edited by TyphooN, 27 January 2015 - 09:32 AM.

  • 0

#12

C.M
  • C.M
  • Crypto-Laird
  • MEMBER
  • PipPipPipPipPip
  • 1,967 posts
  • 100 topics
    • Time Online: 45d 18h 6m 28s

In regards to the doc C.M posted - I would like all of the projects under the "To Remove" tab to be removed for the reasons C.M pointed out besides Collatz Conjecture.  Collatz is back up, and it has been a project that has had a ton of work (and the Gridcoin team has racked up a ton of credits at that project).  I don't particularly care for the science behind the project, but historically it has been one with plenty of work to go around.

 

Currently Collatz Conjecture is showing:

Work # Tasks ready to send 1,583

Tasks in progress 60,305

 

 

I believe we shouldn't add TN-Grid because it requires an invite.  If everyone cant easily sign up, that is not fair game in my book.

 

I don't see any issues with adding the following projects:

* SRBase

* vLHCathome

* Universe@Home

I've converted it back to google docs, and made a lot of changes:

We've now got a changelog/notes page.

I've moved Collatz Conjecture and MindModeling@Home back to the whitelist (I made a mistake with mindmodeling. I thought it had no work units, turns out it's an active BOINC project).

I've added more info throughout the spreadsheet. I'll add each project's individual aims so that everyone knows what the BOINC projects do.

 

https://docs.google.com/spreadsheets/d/1MRNTNGSKykYbJDhrSMuv-P_GjmqkQBCKspaa6CzVeVA/edit?usp=sharing

 

Note: You'll only be able to comment & view the spreadsheet. In order to start editing it yourself, you'll need to pm me your email address.

Make a new gmail account for this sole purpose. I don't want to know your BOINC email address!

 

We should discuss the following whitelist applicants (they've not been decided on yet):

WEP-M+2 Project

Citizen Science Grid

BOINC@Fiit


Edited by C.M, 27 January 2015 - 02:52 PM.

  • 0

^ Smash that upvote button! ;D

Follow me on:

Github

Twitter

Steemit

SoundCloud (Hangouts)


#13

TyphooN
  • TyphooN
  • Veteran Member
  • MEMBER
  • PipPipPipPip
  • 702 posts
  • 2 topics
    • Time Online: 27d 20h 21m 8s
  • Country:United States

Just to clarify my previous comment (in regards to mindmodeling specifically).  I actually like mindmodeling quite a bit, and run it on all my CPIDs but I just notice that work frequently isn't available.  I am personally very interested in the scientific research it wants to produce, but I have just seen several periods of time where work hasn't been available.  We haven't really come up with a definitive list of standards that all projects need to adhere to, but I just wanted to bring up that point (which is why I voted to have it removed).  Obviously we can't guarantee that work is available from projects that we don't control, but I just thought I'd clarify why left it lumped in with the other to be removed projects even though I support the project.


Edited by TyphooN, 29 January 2015 - 02:29 PM.

  • 0

#14

TyphooN
  • TyphooN
  • Veteran Member
  • MEMBER
  • PipPipPipPip
  • 702 posts
  • 2 topics
    • Time Online: 27d 20h 21m 8s
  • Country:United States

I think FiND@Home is a worthy project but it just has a low number of tasks ready to send.  Do you think that tasks available to send is a good metric for enabled projects?  Is there a way that we could disable a project if it goes under 500 (or another arbitrary number) tasks available to send?  I just don't like the fact that there are times that I have had RAC on some projects when work is unavailable to others. If we want a project enabled but don't want to allow a temporary advantage to certain users when work is unavailable this seems like a sensible solution to me.

 

WEP-M+@ Project (wanless2) does not have enough work, can only run on 32 bit OSes, and like you said isn't even sending you work in your VM.  Don't add it.

I also don't believe we should add BOINC@Fiit.  I can't even load the message board page, and it is unsettling to me when I can't even read about the science behind a project.

I am unable to load http://volunteer.cs.und.edu/- until we can research this project I would say that we shouldn't add it.


  • 0

#15

C.M
  • C.M
  • Crypto-Laird
  • MEMBER
  • PipPipPipPipPip
  • 1,967 posts
  • 100 topics
    • Time Online: 45d 18h 6m 28s

I think FiND@Home is a worthy project but it just has a low number of tasks ready to send.  Do you think that tasks available to send is a good metric for enabled projects?  Is there a way that we could disable a project if it goes under 500 (or another arbitrary number) tasks available to send?  I just don't like the fact that there are times that I have had RAC on some projects when work is unavailable to others. If we want a project enabled but don't want to allow a temporary advantage to certain users when work is unavailable this seems like a sensible solution to me.

This is the thing I'm not sure about - what if a BOINC project has a constant buffer of 200ish (for example) work units ready at any one time as to save resources on their systems?

It'd be really handy to keep a history of available tasks for each BOINC project - incase projects go through work availability phases, etc.

I don't believe it's possible for the gridcoin client to dynamically change the whitelist accoring to task availability.

 

There's either two things we could do:

1. Directly contact each BOINC project that has a low quantity of work units & enquire into future BOINC work unit availability.

2. Remove BOINC projects from the whitelist if there are insufficient work units available for the entire team. We'll have to determine what that quantity is, as you say it could be 500+ considering we've got 200+ users..

 

 

WEP-M+@ Project (wanless2) does not have enough work, can only run on 32 bit OSes, and like you said isn't even sending you work in your VM.  Don't add it.

I also don't believe we should add BOINC@Fiit.  I can't even load the message board page, and it is unsettling to me when I can't even read about the science behind a project.

Seconded, I created 2 dedicated VMs for this and it didnt work, gutted.

 

 

I am unable to load http://volunteer.cs.und.edu/- until we can research this project I would say that we shouldn't add it.

I can load it, but I think I previously had a similar issue.. we can put its addition off until others verify whether or not they can access it. http://volunteer.cs.und.edu/csg/ is it the missing /csg/ that's the issue?


  • 0

^ Smash that upvote button! ;D

Follow me on:

Github

Twitter

Steemit

SoundCloud (Hangouts)





0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users