Programming & Scripting

View unanswered posts | View active topics


Post a new topicPost a reply
Previous topic | Next topic 

User avatar   Beginner BeginnerMember since: 11.02.2012, 11:45Posts: 27Location: California Likes: 1
 

 Post Topic: Camera lags when attached to the player.
PostPosted 14.03.2012, 02:03 
The end result that we want is to have the camera effectively attached to a helper on the player model. Among other things, we need to have a user's first person view to be completely consistent with what other users see. This prevents us from using separate view models, so the solution our team has chosen is to use the internal "third person" mode, and simply attach the camera to the character's head.

Now we have had some success following this approach, but we have run into a significant issue. We've been working at it for some time now, but hopefully someone here will be able to help us out.

Our solution to "attaching" the camera is through some code added into playerView.cpp, it basically calls the position and rotation information and then applies it to 'viewParams' every frame.

We've even added some code for an iron sight view, and created a helper in our gun model that is lined up with the sights and we use the same technique to attach the camera to that helper when the player is using iron sights.

The problem is that the camera will lag behind wherever it is really supposed to be, if the camera is always in the same position relative to the object then that object will always be at the same exact spot on your screen no matter what is happening. However that is not the case. When moving or aiming side to side, the camera will become dis-aligned from the gun until you stop moving/aiming.

We've made a couple videos to demonstrate.



You can see the problem in this video, the gun is constantly getting dis-aligned with the camera. The camera should be attached to the exact same point of the gun every frame, meaning that even when aiming or moving around, the gun should not move at all relative to the screen. It appears that viewParams is getting outdated information, maybe the camera is getting the position and rotation value of the gun from last frame instead of the current one? Although It seems more likely caused by some form of Interpolation, which we tried to investigate. However, we didn't get far as the relevant source code seems to be out of reach for the Free SDK.

Here is the code that we use to set the camera's position each frame. When zoomed, it attaches the camera directly to the "ironsight_view" helper located on the gun. When not zoomed, it attaches the camera to the "eye_left_bone" helper on the player model.

Code:
// (Inserted into playerView.cpp, called from the function "ViewProcess")

// Get the player's skeleton as pPose and the player's actor as pActor
ISkeletonPose* pPose = m_in.pCharacter->GetISkeletonPose();
IActor* pActor = g_pGame->GetIGameFramework()->GetIActorSystem()->GetActor(m_in.entityId);
CItem* pItem;

Vec3 pos;
Quat rot;

if( pActor ) {
    // Get the player's Item as pItem
    pItem= static_cast<CItem *>(pActor->GetCurrentItem());
}
// If the players Item is zoomed
if( pItem && pItem->GetIWeapon()->IsZoomed() )
{
    // Get the absolute position and rotation of the ironsight_view helper as pos and rot
    pos = pItem->GetSlotHelperPos(eIGS_ThirdPerson, "ironsight_view", true);
    rot = Quat(pItem->GetSlotHelperRotation(eIGS_ThirdPerson, "ironsight_view", true));
}
//else if not zoomed
else {
    // Get absolute position and rotation for the player's Left eye as pos and rot
    QuatT joint = pPose->GetAbsJointByID(pPose->GetJointIDByName("eye_left_bone"));
    Matrix33 rotM = Matrix33(pActor->GetEntity()->GetSlotLocalTM(eIGS_ThirdPerson, false) * Matrix33(joint.q));
    rot = Quat(Matrix33(pActor->GetEntity()->GetWorldTM())*rotM);
    pos = pActor->GetEntity()->GetWorldTM().TransformPoint(joint.t);
}

// Set viewParams' postion and rotation to pos and rot
viewParams.position = pos;
viewParams.rotation = rot;

return;


While we where trying to troubleshoot we decided to try to do that same thing with the Abrams tank and see if it had the same problem, and to our surprise the problem is nonexistent when the camera is attached to the tank:



The camera in this video is attached to the cannon's origin, you can see the cannon's outline in the center of the screen. As you can see it behaves exactly the way it should: the cannon's outline stays at the exact same spot on the screen not matter how much you are moving/aiming.

Here is the code used to "attach" the camera to the tank's cannon:

Code:
// (Inserted into playerView.cpp, called from the "ViewProcess" function)

// Get the tank's cannon as pPart and check if it's valid
if( IVehiclePart* pPart = m_in.pVehicle->GetPart("cannon") )
{
    // Get the cannon's absolute position and rotation and assign both to viewParams
    viewParams.position = pPart->GetWorldTM().GetTranslation();
    viewParams.rotation = Quat(Matrix33(pPart->GetWorldTM()));
    return;
}


We'd like a solution that allows us to attach the camera directly to the user's character model to the same effect as with the tank, so the camera remains perfectly aligned with the helper it is attached to (whether that helper be on his eye, his helmet, his gun, or even a "video recorder" that's hanging around his neck).




Last edited by RedÐ on 15.03.2012, 09:56, edited 1 time in total.

  Beginner BeginnerMember since: 20.08.2011, 03:03Posts: 26 Likes: 0
 

 Post Topic: Re: Camera lags when attached to the player.
PostPosted 14.03.2012, 03:45 
taken from my UDK Game where i did the same

Perhaps just get the Location and rotation of the iron sight bone thing IN the gun and then assign the camera location X to the weapon bone? by that way it would be perfectly alligned i guess

by that way the camera would always be attached to the player and you wouldnt had to attach & detach it
User avatar   Beginner BeginnerMember since: 11.02.2012, 11:45Posts: 27Location: California Likes: 1
 

 Post Topic: Re: Camera lags when attached to the player.
PostPosted 14.03.2012, 07:46 
Quote by alvarofer0020:
taken from my UDK Game where i did the same

Perhaps just get the Location and rotation of the iron sight bone thing IN the gun and then assign the camera location X to the weapon bone? by that way it would be perfectly alligned i guess

by that way the camera would always be attached to the player and you wouldn't had to attach & detach it


That's exactly what our code is doing already, but the camera is lagging anyways.


User avatar   Uber Modder Uber ModderMember since: 15.04.2008, 19:40Posts: 2321Location: London, UK Likes: 1
 

 Post Topic: Re: Camera lags when attached to the player.
PostPosted 14.03.2012, 16:02 
Note, that each item has it's own camera helper. What you did is okay, but the thing that you call the lag isn't a LAG, just the weapons are moving locally relative the camera, attached to player. You need to find the Camera helpers on each weapon, and then attach that camera exactly to these helpers. Only after you do that, your camera will move together with the weapon.

BUT! It seems like 3rd person weapons aren't made for that. Maybe you should correct these helpers of ironsight, for each shoot animation?
User avatar   Beginner BeginnerMember since: 11.02.2012, 11:45Posts: 27Location: California Likes: 1
 

 Post Topic: Re: Camera lags when attached to the player.
PostPosted 14.03.2012, 19:29 
The helper is parented to the object, I have not animated this helper at all so it is not ever going to move relative to the gun, in fact there are absolutely NO animations for the gun at all. The camera most definitely is lagging, it lags when it's getting it's position and rotation information from the player, like his head or his gun, but it doesn't lag when getting it's info from the tank.


User avatar   Beginner BeginnerMember since: 29.08.2011, 01:17Posts: 10 Likes: 0
 

 Post Topic: Re: Camera lags when attached to the player.
PostPosted 15.03.2012, 02:24 
We made a video to more clearly demonstrate this issue where the geometry and the data retrieved from the helper position are not properly synchronized. The red and blue cones represent the frame-by-frame position and orientation our code gets from the helpers. You can see the issue most significantly at 0:05.



The following code is used to display the cones. It lives in Player.cpp, in the CPlayer::UpdateView function.

Code:
// Get the player character's skeleton pose.
ISkeletonPose* pPose = GetEntity()->GetCharacter(0)->GetISkeletonPose();

// Get the renderer, for drawing cones.
IRenderAuxGeom* pGeom = gEnv->pRenderer->GetIRenderAuxGeom();

// Get the current item's entity.
IEntity* pItemEnt = GetCurrentItem()->GetEntity();

Vec3 pos;
Vec3 dir;

SEntitySlotInfo info;

if( pItemEnt->GetSlotInfo(eIGS_ThirdPerson, info) ) {
   // Check if the item has data.
   if( IStatObj *pStatObj = info.pStatObj ) {
      // Get the gun's world matrix, for converting to world coordinates.
      Matrix34 gunM = pItemEnt->GetSlotWorldTM(eIGS_ThirdPerson);
      
      // Get the helper matrix for "ironsight_view" (relative to the gun).
      // For retrieving position and direction.
      Matrix34 ironM = pStatObj->GetHelperTM("ironsight_view");

      // Get the position, converted to world coordinates.
      pos = gunM.TransformPoint(ironM .GetTranslation());
      
      // Get the direction, converted to world coordinates.
      // GetColumn1() retrieves the direction.
      dir = (gunM * ironM).GetColumn1();

      // Draw the ironsight cone, in blue.
      pGeom->DrawCone(pos, off, 0.02f, 0.075f, ColorB(0, 0, 255, 255));
   }
}

// Get the joint directly from the geometry.
QuatT joint = pPose->GetAbsJointByID(pPose->GetJointIDByName("eye_right_bone"));

// Get the character model's world matrix, for converting to world coordinates.
Matrix34 manM = GetEntity()->GetSlotWorldTM(eIGS_ThirdPerson);

// Get the eye's position, converting to world coordinates.
pos = manM.TransformPoint(joint.t);

// Get the eye's direction, converting to world coordinates.
// GetColumn1 gets the Matrix34's direction.
dir = (manM * Matrix34(joint.q)).GetColumn1();

// Draw the player eye cone, in red.
pGeom->DrawCone(pos, off, 0.02f, 0.075f, ColorB(255, 0, 0, 255));


ImageRedShard Games dev
User avatar   Beginner BeginnerMember since: 29.08.2011, 01:17Posts: 10 Likes: 0
 

 Post Topic: Re: Camera lags when attached to the player.
PostPosted 16.03.2012, 19:39 
Further, here is a screenshot with DebugDraw enabled on the gun entity.

Image

As you can see, this issue does not seem to be related to the camera in any way. The blue cone is perfectly attached to the position of the gun revealed by DebugDraw. We really need to get the position of the geometry/helper as is, the way it is actually represented on screen. The current result, even as a close approximation, is unfortunately not an acceptable compromise for our particular needs.

It goes without saying, but any information regarding this issue would be greatly appreciated. At the very least, we'd like to know if getting the correct position of the geometry is even at all possible in the Free SDK.


ImageRedShard Games dev
User avatar   Uber Modder Uber ModderMember since: 29.05.2010, 11:46Posts: 1326Location: Germany Likes: 163
 

 Post Topic: Re: Camera lags when attached to the player.
PostPosted 16.03.2012, 20:43 
What is lagging is the animation or the position/rotation you attached to not the camera.
The bones do not get updated at once every frame they are interpolated (by linking this to the camera you get a lag)


User avatar   Beginner BeginnerMember since: 29.08.2011, 01:17Posts: 10 Likes: 0
 

 Post Topic: Re: Camera lags when attached to the player.
PostPosted 16.03.2012, 22:00 
Yes, we are fully aware it is not the camera itself that is lagging. I apologize that the title of the topic was misleading.

As was mentioned in one of my previous posts, we have already investigated Interpolation, but have not been able to track down where exactly this is applied in the code, nor how to disable it. It would be extremely helpful if we could get the position/orientation data that perfectly reflects the geometry, as rendered on the screen. If you or anyone else knows how to do this, or perhaps how to disable the interpolation (or recalculate it, to mathematically reproduce the visual position), please share.


ImageRedShard Games dev
User avatar   Beginner BeginnerMember since: 11.02.2012, 11:45Posts: 27Location: California Likes: 1
 

 Post Topic: Re: Camera lags when attached to the player.
PostPosted 17.03.2012, 02:22 
If it is being interpolated then wouldn't the animation be lagging behind the true position? In this case the animation is actually going up ahead of the true position, that doesn't seem like any kind of interpolation to me. unless the true position is being interpolated, but what need would there be to interpolate something that isn't even normally visible?


  Beginner BeginnerMember since: 20.08.2011, 03:03Posts: 26 Likes: 0
 

 Post Topic: Re: Camera lags when attached to the player.
PostPosted 17.03.2012, 14:52 
Just for curiosity the bone is part of the player skeleton? i mean i got the true first person method working well in UDK and im looking if similar concepts could be applied here

Also dident the free Sdk had a event/function to change the player view? like calc_view if it doesnt then i guess you would had to play a bit with the bone properties in both code and in the editor

I have really never coded anything for the free sdk so i could obviously be wrong
User avatar   Experienced Modder Experienced ModderMember since: 30.06.2011, 13:37Posts: 461Location: Ottawa, Ontario - Canada Likes: 5
 

 Post Topic: Re: Camera lags when attached to the player.
PostPosted 17.03.2012, 16:20 
I am no coder so please excuse my comment if this does not apply but I am great with engines and work flows in Animation. I am an Animation grad so here is a thought you may try.

because Engines Are not good at heavy calculation they require most things to be pre simed or cashed before imports into the engine either export the cam Collada information from the engine. import it into the character and parent it to the actual rig.
Or alternatively create a null object say a transparent cube parent that to the skull joint then parent the camera to the Null object in the engine. this Voids the engine calculating the Camera information from the animation and the Engine will sperate the animation clip from the camera data.

best of luck


Image
User avatar   Beginner BeginnerMember since: 11.02.2012, 11:45Posts: 27Location: California Likes: 1
 

 Post Topic: Re: Camera lags when attached to the player.
PostPosted 19.03.2012, 00:02 
How would I go about linking the camera to a bone? I've messed around with he flowgraph stuff and I've been able to link the camera to the player entity itself using entity:AttachChild:

Clipboard02.png

This FG works, but it makes the camera attached to the player's entity, which results in the view being at the player's feet. I can't find a way to get the entity ID of an actual bone to put it into AttachChild. But when I was looking around I found Entity:Attachment which seems to be the same thing as AttachChild only with a "BoneName" field in it. But I can't get this node to work:

Clipboard01.png

This FG does nothing whatsoever, and when I change the input to activate the Camera:View it just shows the camera position where I placed it on the map, meaning Entity:Attachment is not working at all. Am I using it wrong?


User avatar   Beginner BeginnerMember since: 11.02.2012, 11:45Posts: 27Location: California Likes: 1
 

 Post Topic: Re: Camera lags when attached to the player.
PostPosted 19.03.2012, 07:03 
I was able to link the camera to the weapon's entity using Game:WeaponSensor and Entity:AttachChild, I was unable to attach it to a bone though, just the entity itself so the camera is clipping through the gun. Unfortunately this revealed that even when the camera is linked instead of getting it's position set every frame like our code was doing it still acts exactly the same:



For some reason it seems that the true position (The position revealed by DebugDraw like in an earlier post) is being interpolated to the position of the actual animated geometry; but that is very peculiar because the true position isn't visible to the player normally, why would it ever need to be interpolated?


User avatar   Uber Modder Uber ModderMember since: 15.04.2008, 19:40Posts: 2321Location: London, UK Likes: 1
 

 Post Topic: Re: Camera lags when attached to the player.
PostPosted 19.03.2012, 13:22 
Scale the vector of camera position, just move it forward on direction.