posted by sokhaty

posted by sokhaty

Oh, and your boss will know exactly how long did you spend in front of the monitor looking at the code versus watching videos on C9 [blink, blink]

posted by sokhaty

So, was Steve making some sort of a point by playing only A major? Azure-Majure?

posted by sokhaty

For similar reasons, in order to pick a better algorithm, one has to disclose all the constaints and assumptions used to compute measure of "goodness".

posted by sokhaty

It would be very helpful to look at some samples. Also, it looks like the discussion is based on embedded assumption that observations of the events are direct. I wonder how it all change if observations were indirect - for example when you are trying to reconstruct flow of events based on gathering clues. Stream of clues is directly observable (and ordered in time), while pointers to the event stream of interest can be to the past or future times or be just durations without definite starting time points. Would Rx be useful in this case?

posted by sokhaty

I guess a real take away from this episode is how easy it is to mess thing up big, whithout understanding what events are and how they are being processed at run time.

posted by sokhaty

And it's rather obvious that "like.. you know" Bart now works for Eric.

Just out of curiosity, what is so special about SharePoint security that computation has to happen in the business layer as opposed to a stored procedure or even a correlated subquery in a SQL statement?

posted by sokhaty

I'd like to hear more about developments in the area of type systems, relational (im)purity,  and programming languages to replace SQL.

Cost based optimization and how things are evolving in that area would be interesting too.

On a side topic, calling SQL relational is a blasphemy. It's as relational as a cow is a noble steed.

posted by sokhaty

Is it the same or similar reason why guys from Jane Street Capital hint at .NET GC not being quite good enough to meet demands placed on run-time by a functional programming language (F#)?

It looks logical to assume that any engineering solution (software or otherwise) has its range of applicability. Going below or above the applicable range requires some other engineering solutions. Is there any info out there on applicability limits of .NET GC and .NET VM for that matter (and how one would express those limits anyways - in terms of lattency or memory allocations per unit of measure, when being general purpose VM starts and ends?).

posted by sokhaty

(I guess inclusion of MDM services into 2008 R2 may seem to hit that the answer is "soft of").

To the subject of the code running closer to the data, this was obvious for quite a while now that shipping bits over network to have a copy in application memory and process it using (more often than not) less sophisticated algorithms than those already present at the point of storage doesn't make much sense. Database is a computational engine, so it should have a powerful language to create computational expressions. Unfortunately SQL in general is not quite that, and T-SQL in particular isn't either.

Having F# inside database engine would be nice. Add "pure mode" for querying (which should be side effects free) and use imperative mode to produce side effects (data modifications).

posted by sokhaty

Then instead of add one before, remove one after there will be explicit call to ce.NoMoreCounterIncrements()

It doesn't guarantee that programmer won't forget to add it into the code, but that's one change to one code line as opposed to two (and it's a better "pattern" than the alternative).

Plus, if this primitive constructor accepted nested lambdas, then AddCounter() and NoMoreCounterIncrements() could be hidden from the programmer all together. Just declare what you want to spawn and how many of those in a nested lambda and happily wait for completion.

posted by sokhaty

It seems that an adanced enough run-time can and should use both, based on the accumulated "knowledge" (stats) about workloads being executed.

Expectation that something can be strictly evaluated in false in absolute sense, because each and every CPU instruction and/or memory read/write may fail because of faulty hardware. Yet, it can be statistically true. If hardware is somehow known to be 99.something% reliable, such assumption can be made safely (in statistical sense), otherwise nothing can be computed or done ever.

(I believe that proponents of strict evaluation are stuck because they base their reasoning on incorrect assumptions without explicitly stating what those assumptions are, which is a known issue that plagued physics for centuries, and most likely still does)

The same must apply to the algorithms as well. If algorithm is known to be predictable on a given workload (either statistically or by devine intervention of the mister human), it's OK to evaluate is strictly. If there is no prior knowledge, lazy evaluation is the way to go and please gather execution stats upon exit so it can be reused in the future evaluations/executions. And if it does not exit in the requested amount of time - abandon (preferrably kill first) the execution and and black list it (till the end of time or the next devine intervention).

From 10000 feet it looks like a nice logical schema with a feed back loop, which is statistically a necessity for each and every successful eco system (observe the nature).

posted by sokhaty

In the end it will be about believes, whether one believe this or that language being "new" or not. Attempts to define "pure novelty" would end up nowhere.

posted by sokhaty

I guess that the counter is NULL originally (if not set to any positive integer).

posted by sokhaty

In order to run a program on a parallel hardware, run-time would have to reason about side effects to come up with some strategy to partition computational graph into work loads that have minimal interactions between each other.

If many core processors will have cores of different capabilities (which seems to be the case), run-time reasoning and JIT will be a necessity.

It seems like none of the existing imperative languages would survive transition to parallel era. Of course run-times are still be written in something that is sugar coated assembly, yet for general-purpose programming completely new languages would be required.

Declarative and richly typed presumably.

Also to the point of run-time reasoning and code generation, to provide fault tolerance computational graph might need to be re-evaluated if a computation node returns exceptional value or goes into non-termination state. That in theory would allow automatic remediation for run-away queries in databases and handling of non-responding services in the cloud (as well as mutating hardware - failed or hot plugged general and special purpose CPUs, failed or hot plugged memory and so on).

It probably will take another 10 to 20 years to get it right, but it looks like that's where things are going.

posted by sokhaty

As long as ce.AddCount() is inside foreach loop on the main thread there won't be any race conditions, so adding "one for the host" before and then taking it out after the loop is kind of pointless.

Also, guys, _No Sound_ in SL3 on XP SP3 _again_.

Cheers,

Seva.

posted by sokhaty

If 90% of the input for my app on any given day happens to be small, it's better to process those 90% sequentially and use parallel execution only when appropriate, but for that it would be nice to know, where (approximately) is that cutting point.

I can run tests and collect some stats on what overhead of firing up parallel execution is, but assuming that this work might have been already done while developing the parallel framework, it would be preferrable for me to look at the stats collected by the PF development team than spend time and efforts myself.

Cheers,

Seva. 

posted by sokhaty

One thing though, when matrix multiplication was executed on a small data set, sequential was actually faster than ParallelFor (@20:48). Are there any insights on estimating an overhead of setting up parallel execution machinery, so, application could attempt guessing whether to process data set sequentially or in-parallel (assuming that application knows the size of the data set)?

posted by sokhaty

posted by sokhaty

Why is it opening compressed folders in Windows Explorer is so slow?

Why Windows (7 included) allows me to create files and folders with names in (practically) any national alphabet, but doesn't allow me to send them to a compressed file?

posted by sokhaty

posted by sokhaty

posted by sokhaty

It surely addresses a lot of potential issues with measurement mismatch. 

Yet, run time support for measurements still makes a lot of sense.

If you are reading data from external sources at run-time (files, sensors, or web services), you'd still have to implemet all the measurement tracking and conversion yourself. If this can be married to contract somehow, then application would just have to tell the framework that it expects mass to be in kilos, and if input feed turns out to be pounds then conversion would happen under the scene.

Another presumably useful feature would be to declare measures off of classes/types. If for example, I'm counting my chicken, I don't want to be able to inadvertently add this to the count of eggs, unless I explicitly coersed chicken and eggs to be "things".

Anyhow, compile time support is a very good start.

posted by sokhaty

Now, this in-memory columnar database, is it going to be shipped along with Gemini only, or it eventually make it into some edition of SQL Server engine?

Sure, database query language doesn't have to be SQL (or arguably even shouldn't be SQL), but querying language aside, such a compelling feature as in-memory columnar storage  looks very appealing as a generic service.

posted by sokhaty