page 1 of 2
Comments: 25 | Views: 212
harlock123
harlock123
Tall, Dark, and Gruesome

Take this simple code snippit

            List<NJTrackingEntity> result = new List<NJTrackingEntity>();
            try
            {
                SqlConnection cn = new SqlConnection(DBCON());
                cn.Open();
                string sql = "select A.SPECIALPOPULATION as 'CODE'," +
                             "COALESCE(B.DISPDESC,(SELECT TOP 1 C.SITENAME from TblResidentialProvLoginsPgm C " +
                                "LEFT OUTER JOIN xvt_userauthprograms D on " +
                                "C.USERID = D.USERID AND D.PGMINDICATOR = A.SPECIALPOPULATION " +
                                "WHERE C.USERID = D.USERID )," +
                                "(SELECT TOP 1 C.ORG from TBLUSERLOGINS C " +
                                "LEFT OUTER JOIN xvt_userauthprograms D on " +
                                "C.USERID = D.USERID AND D.PGMINDICATOR = A.SPECIALPOPULATION " +
                                "WHERE C.USERID = D.USERID )) as 'DESCRIPTION', " +
                             "A.EFFECTIVEDATE,A.EXPDATE,A.MSPID " +
                             "from dbo.tblMemberSpecialPopulation A " +
                             "LEFT OUTER JOIN dbo.LOOKUPTRACKINGELEMENTS B ON " +
                                "A.SPECIALPOPULATION = B.Code " +
                             "where MEMBERID = @P1 " +
                             "ORDER BY EFFECTIVEDATE DESC,EXPDATE DESC";
                SqlCommand cmd = new SqlCommand(sql, cn);
                //cmd.Parameters.Add("@P1", System.Data.SqlDbType.Int,4,"MEMBERID").Value = Convert.ToInt32(mid);
                cmd.Parameters.Add("@P1", System.Data.SqlDbType.Int, 4, "MEMBERID").Value = mid;

                SqlDataReader r = cmd.ExecuteReader();
                while (r.Read())
                {
                    NJTrackingEntity nj = new NJTrackingEntity();
                    nj.ID = r[4] + "";
                    nj.CODE = r[0] + "";
                    nj.DESCRIPTION = r[1] + "";
                    if (!r.IsDBNull(2))
                        nj.EFFECTIVE = r.GetDateTime(2).ToString("yyyy/MM/dd");
                    if (!r.IsDBNull(3))
                        nj.EXPIRATION = r.GetDateTime(3).ToString("yyyy/MM/dd");
                    result.Add(nj);
                }
                r.Close();
                cmd.Dispose();
                cn.Close();
                cn.Dispose();

            }
            catch (Exception ex)
            {
                NJTrackingEntity nj = new NJTrackingEntity();
                nj.DESCRIPTION = ex.Message;
                result.Add(nj);
            }

            return result;

 

For some sets of memberIDs handed to this I get database timeout errors and for others it works just fine.

Pulling my hair out looking for database tuning bits and what not to make the timeouts go away. I though the only thing

I could do was to increase the timeout values on the command object. No matteer what I put there though the timeouts persisted

of course taking longer and longer to return to me the error result.

 

The Query however take out of code and plaaced into Query tools in SQLDev Studio executed quickly and without error.

 

Again this only occured with some MemberID values.

 

What I found was very surprising change the code to also include a simple addition to the where clause of 'and 1=1' make the issue

disappear.

 

So the execution engine for sql commands in .Net was creating some sort of issue that it should not have been doing?

Makes me wonder where else in the code this kind of crap might be happening....

 

I sometimes get strange query results in Oracle if I don't specify the tables for all of the fields. Which table is MEMBERID in? It might be confusing the execution engine.

 

Anything special about the member IDs that cause the command to timeout?

Blue Ink
Blue Ink
C you

As far as I know, the .NET engine does nothing special to your command... it should only format the request and pass it on to the server for processing.

 

What I believe might be the problem here, is the one that goes under the name of "parameter sniffing". Most of the documentation you can find on the subject is about stored procedures (that's how I found about it), but it applies also to any query that could be effectively cached.

 

In a nutshell, what happens is this: whenever you send a query to SQL, the query gets compiled. Since that takes time, SQL server tries to cache the queries it compiles so it doesn't have to do it every time. So far so good.

The rub is that the execution plan of the query gets optimized using the parameters that are passed the first time around. These values may lead SQL Server to create an execution plan that is highly optimized for a trivial case, but that is simply awful with different values. This also explains why the query gets apparenty "fixed" by inserting a silly condition like "AND 1 = 1": since the query changed, the cached query can no longer be used and a new plan is created and that could be much better than the cached one.

 

EDIT: this might also explain why the query behaves differently interactively: even if the query is the same, you probably passed in the values directly instead of going through parameters. That should be enough for SQL Server to see it as a totally different query.

 

You may want to add some query hinting (I'm thinking of OPTIMIZE FOR UNKNOWN, but I don't know if that applies to your case).

Does SQL Server cache queries without looking at parameter values? That surprises me, since parameter values will likely change and would even lead to bad results and/or inefficient execution plans.

Blue Ink
Blue Ink
C you

The problem is that it does look at the parameters that are passed at the time the query is cached. There are heuristics that can trigger a recompilation of a cached query, but like all heuristics they are not foolproof.

 

Some nitty-gritty details are in the technet article I linked earlier (the full title is Batch Compilation, Recompilation and Plan Caching Issues in SQL Server 2005).

 

figuerres
figuerres
???

there is one known bug in .net's sql parameters that you might be hitting - possibly.

 

can that int ever be zero?

 

in cases where a zero is passed it cam be mis-applied as a different parameter.

like data size or data type.

 

i have a place in one of my apps that has a work around where it has hit me.  but it takes a very specific thing to trigger it.

 

my other question would be to run the query and get the time it takes to execute if it takes more than about 10 seconds then i would use query analyzer to find out why.

 

i suspect you may have some perf issues in there that should be addressed.

 

i have seen a first pass at a query take 30 seconds and then an index drop the same query to less than one second.

 

generaly any query that needs more than 5 seconds to run in query analyzer would be suspect to me.

figuerres
figuerres
???

there is a bug in the parameters object that can mis-read a zero int value in some cases. i do not think you are getting that bug but just in case i felt it was worth mentioning.

 

in my case i had a param that had to be zero, and when i used an actual 0 in the parameter list it would freak out.

i had to create a const int Zero = 0;

and use that as my parameter. 

I would use query ana. and find out what's eating up the time....  adding an index or a view as needed... or re-writing some of the sql possibly... i'd have to have a sample to test with to know for sure.

figuerres
figuerres
???

but whats the execution time in QA ? 

what does the plan show for what branches take the most % of the total time?

are the major points getting an Index Seek? Index Scan? or other?

 

figuerres
figuerres
???

also try making a sproc out of it and see what happens if you call that in QA and from .net

Sven Groot
Sven Groot
My name has 9 letters. Coincidence? I think not...

I think you may be misusing the "sourceColumn" parameter of the SqlParameterCollection.Add method. What happens if you change that line to:

 

cmd.Parameters.Add("@P1", System.Data.SqlDbType.Int).Value = mid;

 

(I also removed the size as it is not necessary for SqlDbType.Int).

figuerres
figuerres
???

Ya know that does look odd, i never did it that way.... looking at one of my code blocks i have done it 2 ways and nither look like his.

 

here is one way i have done that works unless it's a zero:

command.CommandType = CommandType.StoredProcedure;

command.Parameters.Add(new SqlParameter("@FranchiseID", FranchiseID));

 

that is one way...

 

another is that i would create each param and then make the collection so that i could do a loop where the params chnaged with each pass but no creating objects inside the loop unless i had to.

 

like :

 

parameter p1,p2,p3;

 

params.add(p1)

 

for (...){

p1=somevalue;

cmd.exec;

... and so on.

}

 

you might also be exhausting your connections (either locally and/or on the server) since they are not properly closing on errors.

 

Try creating your sql connect/command/reader (s) inside of a using statement so they get automatically disposed.  Otherwise, put the .Close inside of a finally statement (which is what using does under the covers anyway). 

 

Note:  This does not directly fix your timeout issues.

figuerres
figuerres
???

also true....

 

all my recent sql code (in the last 2 /3 years)  uses using(){} 

in fact it generally is a set of them:

 

using ( sql connection ....){

  .Open();

     using ( sql command ....){

           using ( Sql data reader ...... Cmd.ExecuteReader()){

                while (reader.......){

                }

           }

      }

   .Close();

}

 

with a try block where it seems best in a given case....

 

in a V1 code beta  i did way way back on early 2005 that was not doing this i had connection issues.

in a revised version where i used this pattern of the using statement plus other cleanup no connection limit issues.

the first beta version was very very rapidly built to get the project started, we knew it had issues to fix etc...

it used like 260 connections! 

when the connection use was fixed it now never uses maore than the default connection pool limits.

 

and runs today with about 300 client machines connecting to it.

figuerres
figuerres
???

this may sound odd but...

 

Use using, it uses Dispose when your using goes out of scope to cleanup and stop using the object in question.

 

like the Try / Catch / Finally  says that finally will always happen even if a catch is invoked;  a using always does object cleanup when the used object goes out of scope. so you will not have to write code to make sure that the objects close / dispose logic fires.

 

so use using to close and dispose of any class that impliments the IDisposable interface.

it works and makes it a very clear pattern in the code for others to follow and have correct code.

 

IMHO anyway Smiley

figuerres
figuerres
???

Ok so then under some condtion the query gets a fault.  what is different about the members?

possibly the subquerys fail on select members due to the difference?

 

might that be the issue?  like it cant find a row in that nested part that starts with "COALESCE(B.DISPDESC,(SELECT TOP"

also what pool? connection pool?

 

is that a side efect of the query failure? not closing connections clean on the error?

figuerres
figuerres
???

OK, Next idea to figure this out is to run a trace.

 

I know that parameters and the sql do get sent to the server with some chnages ......

 

Hey hold on ......   i wonder if @P1 is used by the net to sql processing internally.

 

try chnaging that to say  @MemberParam  and see what happens  ...  I think when i do a sql trace i see params beeing done as p1 p2 ... p99  in the sql text

 

might be mistaken but check that.

 

also like i said use sql trace to capture the traffic from your app and see what .net is actualy sending over the wire.  if it is parsing the sql and doing something like the @p1  thing you will see it.

also you may see an error in the sql reply that was hidden by the .net layer.

 

see if that might be it though. or what the sql trace tells us.

page 1 of 2
Comments: 25 | Views: 212
Microsoft Communities